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ABSTRACT 

The research and evaluation agenda for the Austin 
Independent School District (AlSD) (Texas) is determined for each 
school year, subject to current needs and requests. The evaluations 
and other major projects for 1993-94 will focus on three major areas. 
First is providing school support. Testing programs mandated by state 
law and district policy will be coordinated and administered, and 
information and support will be provided to schools. Second is 
research and analysis. Tools for evaluation, such as the AlSD Generic 
Evaluation System (GENESYS) and other district techniques, will 
continue to be developed, maintained, and improved. Using them, some 
long-range analyses will be performed. Third is the evaluation of 
locally funded programs, treatments, and initiatives with a high 
priority for the school district. Program effectiveness studies are 
to include cost effectiveness when possible. Programs funded by 
resources outside the AlSD budget, such as Federally funded and 
state - funded programs are also to be evaluated. New directions that 
are influencing the allocation of priorities and resources of the 
Office of Research and Evaluation (ORE) are (1) accountability, (2) 
the 1993-94 District Improvement Plan, and (3) the study of dropouts. 
Three attachments provide information about ORE staff and operations 
and three figures il lust rate the discussion. (SLD) 
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THE RESEARCH AND EVALUATION AGENDA FOR AISD-1993-94 

EXECUTIVE SUMMARY 

The Research and Evaluation Agenda for the Austin Independent School District is determined through an 
interactive process involving the Board of Trustees, the Superintendent, the Management Team, the Evaluation 
Advisory Committee, the staff of special programs, and other AISD personnel. Although the activities of the 
Office of Research and Evaluation (ORE) constantly adjust to current needs and requests, a set of evaluation 
plans is prepared at the beginning of each school year as a means of obtaining agreement among decisionmakers 
that the proper and most critical infoimation needs are being addressed. These plans provide the blueprints for 
the evaluation staff to follow. 

The evaluations and other major projects for 1993-94 will focus resources in three major areas. 

1. School Support 

Testing programs mandated by state law and District policy will be coordinated and administered, and 
information and support will be provided to schools. These testing programs include the Norm- 
referenced Assessment Program for Texas (NAPT), the Iowa Tests of Basic Skills (ITBS), the Texas 
Assessment of Academic Skills (TAAS), and advanced placement tests (for skipping a grade in AISD). 
Surveys of professional and administrative employees and of former students will continue to be 
administered. 

2. Research and Analyses 

Tools for evaluation, such as the GENeric Evaluation SYStem (GENESYS), the Report On School 
Effectiveness (ROSE), and the Implementation Index will continue to be developed, maintained, and 
improved, and long-range analyses will be performed. Important accountability statistics will be 
maintained and reported on dropouts, at-risk students, retention, and mobility. An enhanced GENESYS 
will be employed to obtain outcome information from District data bases for students in identifiable 
groups such as those in magnet programs, dropout prevention programs, and many others. 

*. Evaluation 

The impact of locally funded programs, treatments, and initiatives with a high priority for the District will be 
evaluated. Program effectiveness comparisons, which include cost-effectiveness information, will be 
reported for a wide range of special programs. Dropout prevention programs for overage ninth graders 
designed for the four operations areas will be examined. Evaluations of two externally contracted dropout 
recovery programs will be conducted. The District's programs for limited-English-proficient (LEP) students 
also will be monitored. In addition, TAAS tutorials, extended school-time efforts in AISD, and the use of 
technology in instruction will be evaluated. ORE will contribute to the Distributive Information Systems for 
Campuses (DISC) project by superintending the refinement and production of profile reports and information 
for the District's annual performance report (APR). 

Programs funded by federal, state, or other resources beyond the local budget of AISD also will be 
evaluated. The success of students receiving compensatory education and other special services will be 
assessed for Chapter 1 Programs, the Chapter 1 Migrant Program, Chapter 2 Formula Programs, the 
Title VII Bilingual Education Transition Program, Drug-Free Schools Programs, and the National 
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Science Foundation (NSF) Grant to the Science Academy of Austin. The effectiveness of training provided to 
teachers through Title II Mathematics and Science Teacher Training also will be examined. 

The 1993-94 budget for the Office of Research and Evaluation reflects a staffing level of 18.25 positions~3 
administrators, 10.25 professionals, and 5 classified staff~a reduction of one local position that was previously 
carried in the Instructional Technology budget, and a reduction of six local positions since 1987-88. Attachment 
A contains a breakdown of ORE staffing levels from its first year, 1973-74, through 1993-94. The loss of staff 
has limited ORE's collection and reporting of process (implementation) information and hampered our ability to 
analyze and to explore why student outcomes resulted as they did. ORE still delivers strong outcome evaluation- 
-especially with the development and refinement of GENESYS, our "generic" evaluation system. 

The ftill Agenda document presents all the programs and requests included for study in 1993-94, the resources 
allocated to each, and evaluation or management plans for the evaluations and management studies adopted in the 
final Agenda. The Table of Contents to the Agenda lists evaluation and management plans for 20 programs, but 
ORE performs many other activities and services not represented by evaluation or management plans. These 
other activities range from meeting with parents to explain their child's test scores, to meeting with school 
faculties to assist them in planning and staff development, to responding to requests for information by AISD 
staff, community agencies, external researchers, university students, and others. These assistance and service 
activities are described in the "Other Activities" section of the Agenda. 

New Directions for 1993-94. 

Three new directions will influence the allocation of ORE priorities and resources this year. 

1. Accountability 

ORE will support AISD's Accountability Support System by providing accountability information for 
schools and the District and by strengthening the liaison relationship with the District's four 
operations areas begun in 1992-93. Last year, an ORE "service representative" was assigned to each 
of the areas. This year, as part of the areas* Service Facilitation Teams, ORE will be able to 
increase coordination in delivery of services to schools. 

2. 1993-94 District Improvement Plan 

The 1993-94 District Improvement Plan specifies several major activities involving ORE, most 
notably as related to the priority areas of "Student Outcomes in Grades 6-9," "School Based 
Improvement (SBI)," and "Professional Development." Several externally funded evaluations have 
components which address improvement efforts in these areas. ORE will continue to monitor student 
outcomes through its annual reports on student achievement, including the Report On School 
Effectiveness (ROSE), and school, area, and District profile reports. ORE also will be evaluating 
some new dropout prevention and dropout recovery programs (see below). 

3. ProPQUtt 

ORE will continue its historically strong leadership role in the study of dropouts and at-risk students in 
1993-94 by evaluating area-based programs to reduce the number of overage grade 9 students and by 
evaluating two dropout recovery programs contracted by the District to community-based organizations. 
ORE will also continue monitoring and reporting on the numbers of at-risk students and dropouts. 
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INTRODUCTION 



The Austin Independent School District has made a commitment to research and evaluation for the past 20 years. 
The mission of the Office of Research and Evaluation (ORE) is to provide objective, accurate, and timely 
information to decisionmakers. The information can range from an individual student's test scores to evaluation 
reports on instructional programs. Decisionmakers car be as different as a parent concerned about a child's 
achievement, an administrator making a programmatic decision, a Board member making a budget decision, and 
a federal funding agency. 

ORE provides an independent perspective on AISD programs as guaranteed by Administrative Regulation BL-R 
(Local) which states that ORE "has the independence necessary to assure unbiased, forthright reports." That 
independence is evidenced by ORE's control over the selection of staff, the administration of funds, and the 
content of evaluation plans and reports. ORE'S mandate is reinforced by Administrative Regulation EAB (Local) 
which requires each campus to participate in "accountability and evaluation activities required by TEA, local 
policy, special program fimding agencies and approved evaluations/ 

The following goals emphasizing service, integrity, and excellence have been set by ORE staff for the 1993-94 
school year: 

Service: • Respond to school and staff requests for service, provide the information that clients need, 
and stay close to the customer 



• Limit paperwork 

• Support AISD's Strategic Plan 

• Improve program effectiveness reporting to facilitate decisions made by the 
School Board 



Integrity: • Keep students* needs foremost, and insure that ORE products and activities contribute to 
student learning 



• Ask the difficult questions, continue to report fully and honestly, and 
maintain autonomy in reporting 

• Improve communication among ORE staffs, within ORE staffs, between 
ORE and other administration staffs, and with school and program staffs 

Excellence: • Work smarter, not harder by anticipating information needs, planning well 
in advance, using participatory management when practical, continuing to 
be in the vanguard in computer hardware and software use, using computer 
resources fully and effectively, and planning activities within resources and 
other constraints 



The purpose of this volume is to outline the activities on which the Office of Research and Evaluation will focus 
in 1993-94. Three key areas will receive special attention this year: 



• School Support 

• Research and Analyses 

• Evaluation 
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Setting the Agenda 



The process for setting the 1993-94 Agenda is outlined below. 



June-November 1993 ORE reports from 1992-93 are reviewed by staff, Management Team, and Board of 
Trustees 

June-November 1993 Funding of external programs and grants is determined. 
August 1993 Local budget resources are approved. 
September 1993-January 1994 Agenda is planned, and proposed allocation of local ORE staff is reviewed. 

September-October Allocation of local ORE staff to projects is planned. 

October-November Evaluation plans are drafted and are reviewed by ORE staff, 
program staff, and Management Team. 

September-January Recommended allocation of local ORE staff is reviewed by: 

September 27 Evaluation Advisory Committee 

Dec. 13, January 3 Superintendent and Management Team 

January 10 Board of Trustees 

Js;;.uary 1994 Agenda is finalized based on Board priorities for allocation of local ORE staff and on 
review and comment on evaluation plans. 



Setting the Agenda for the 1993-94 school year has been difficult because of delays in receiving testing data 
needed for completion of the 1992-93 evaluations, and because of the time required for support related to the 
implementation of the new TEA and AISD accountability systems. Therefore, a functional change from previous 
years was implemented. After soliciting suggestions from the superintendent and the Management Team and 
reviewing requests from various sources for evaluation, ORE developed a proposed allocation of local ORE staff 
resources. (The activities of grant-funded ORE staff are tied to the evaluation of particular special programs, 
and those staff resources are mostly unavailable for the examination of District-funded programs and projects.) 
Then, instead of receiving a nearly finished Agenda for review, the Board of Trustees was asked to review the 
recommended allocation of locally funded ORE staff to priority evaluation activities before an Agenda was 
finalized. This procedure makes the final Agenda more responsive to the information needs of decisionmakers 
and helps to ensure that systemwide priorities are addressed. 

Once priority evaluation activities for local staff were decided and the funding of external programs and grants 
was determined, resources were allocated to the selected projects. The charts " "igure 1 on page 3 represent 
the distribution of resources, in terms of dollars, allocated to locally and externally funded evaluation activities. 
In 1993-94, 60% of ORE's budget is from local revenue and 40% is from external sources. Allocation of ORE 
local staff resources is shown in Figure 2. 

The limited availability of staff and other resources necessitates the careful selection of tasks for ORE; therefore, 
requested evaluation activities with high priority for the District replaced other requested evaluation activities 
with lesser priority. Charts detailing the requests for local ORE staff resources against the available resources 
are contained in Attachment B. As shown in the attachment, total requested evaluation activities required an 
estimated 23.20 full-time equivalent (FTE) staff, while only 13.60 FTE staff were available (11.0 regular staff 
+ .60 staff from externally funded programs contributed to District activities + 2.0 temporary staff). 
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FIGURE 1 

1993-94 ORE BUDGET FROM LOCAL AND EXTERNAL SOURCES 



Local Budget 

$604,400 (60%) 



School Support and 




Program Evai. (10%)* 

♦ Program Evaluation fodud#3 evsJuetfon of program* for LEP students, •valuation of dropout prevention programs for 
overage 9th grader? and of two contracted dropout recovery program*. assessing the coat*effecUvene*a of programs, 
and the supervision of smeJ, externally funded evaluations. 



External Budget 

$394,957 (40%) 



Migrant (6%) 



Chapter 2 (8%) 




Drug-Free (10%) 
EP (1%) 

Title VII (4%) 
^-NSF (3%) 
-^Title II (2%) 
^Rlver Watch (1%) 





Chapter 1 (65%) 
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FIGURE 2 
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Note: -ORE's budget currently includes 11 positions plus funds for temporary staff equivalent to 2.0 full-time equivalents (FTEs). Total = 13.00 
-Federal funds (EXT) can contribute 10% of resources for districwide activities (.60 FTEs). Total for ORE = 13.60 
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Figure 3 on page 6 is a chart of the 21 items that were selected for inclusion in the 1993-94 Agenda based on 
available resources and on a review of the projects 1 characteristics, Attachment B lists all the items considered 
including those for which evaluations resources were requested but not allocated. Attachment C describes each 
activity selected in terms of: 



Mandate: 



Evaluobility: 



Utility: 



Required/requested by School Board? 

Required by State (law or SBOE rule)? 

Required by external funding agents 
(state/federal)? 

Requested by superintendent/administration? 
Requested by divisions/departments/schools? 
Evaluation need identified by ORE? 
Process evaluation (implementation): 
Possible? 

Recommended by ORE? 

Product evaluation (achievement/attendance/behavior/other): 
Possible? 

Recommended by ORE? 

Does it provide new/useful information? 

Is there potential for findings being used: 

Budgetary? 

Instructional? 

Can data be provided when needed? 
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FIGURE 3 



ORE AGENDA-SETTING PROCESS 1993-94 
Agenda Items for Which Resources Are Available 



Program/Evaluation Activity 
Dropout/ At-Risk Statistics 



Student Achievement 
(TAAS, NAPT, ITBS) 



Former Student Survey 



78702 RAYS Dropout Recovery 
Program 



American Institute for Learning 
(AIL) Dropout Recovery Program 



Program Effectiveness Comparisons 



Historical Student Retention in Grade 



Programs for LEP Students 
Chapter 1 
Chapter 1 Migrant 
Chapter 2 



Mandate 

Required/requested by School Board 
Required by State (law or SBOE rule) 
Requested by superintendent/administration 
Requested by divisions/departments/schools 
Evaluation need identified by ORE 

Required/requested by School Board 
Required by State Oaw or SBOE rule) 
Requested by superintendent/administration 
Requested by divisions/departments/schools 

Required/requested by School Board 
Requested by superintendent/administration 
Requested by divisions/departments/schools 

Required/requested by School Board 
Requested by superintendent/administration 
Requested by divisions/departments/schools 

Required/requested by School Board 
Requested by superintendent/administration 
Requested by divisions/departments/schools 

Required/requested by School board 
Requested by superintendent/administration 
Evaluation need identified by ORE 

Required by State (law or SBOE rule) 
Requested by superintendent/administration 
Requested by divisions/departments/schools 
Evaluation need identified by ORE 

Required by State (law or SBOE rule) 

Required by external funding agents 

Required by external funding agents 

Required by external funding agents 
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Program/Evaluation Activity 
Drug-Free Schools 

National Science Foundation (Science 
Academy of Austin) 

Title II Mathematics and Science 
Teacher Training 

Title VII Bilingual Education 
Transition Program 

Distributive Information Systems 
for Campuses (DISC) 

TAAS Tutorials 



GENeric Evaluation SYStem 
(GENESYS) 



Technology for Instruction 



Extended School Time 



Overage Ninth-Grade Student 
Dropout Prevention Programs 



FIGURE 3 (Continued) 

Mandate 

Required by external funding agents 
Required by external funding agents 

Required by external funding agents 

Required by external funding agents 



Requested by superintendent/administration 
Requested by divisions/departments/schools 
Evaluation need identified by ORE 

Requested by superintendent/administration 
Requested by divisions/departments/schools 

Requested by superintendent/administration 
Requested by divisions/departments/schools 
Evaluation need identified by ORE 

Requested by superintendent/administration 
Requested by divisions/departments/schools 

Requested by superintendent/administration 
Requested by divisions/departments/schools 

Requested by superintendent/administration 
Requested by divisions/departments/schools 
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HIGHLIGHTS 

The 1993-94 Agenda reflects the following important activities for ORE. 

• Supporting AISD's Accountability Support System by providing school and District accountability 
information and by serving on area Service Facilitation Teair* 

• Continuing support for School Based Improvement (SBI) through ORE "service representatives" and 
technical assistance with campus improvement plans (CIPs) :ind Chapter 1 improvement plans 

• Coordinating expanded testing activities related to the administration of the Texas Assessment of 
Academic Skills (TAAS) at grades 3-8 and 10 (Exit Level) 

• Coordinating new end-of-course tests in Algebra I and Biology I 

• Continued coordination of the administration of the Iowa Tests of Basic Skills (ITBS) at grades 1 and 2 and 
the Norm-referenced Assessment Program for Texas (NAPT) at grades 3-11 

• Continued monitoring of student outcomes through annual reports on student achievement, especially the 
Report On School Effectiveness (ROSE) 

• Continuing support for schools by increasing data access and data manipulation capabilities at the campuses, 
Distributive Information Systems for Campuses pISC) 

• Production of revised and improved school and individual profiles 

• In-depth evaluations of Integrated Learning Systems (Computer Curriculum Corporation and Jostens 
Learning) at Chapter 1 schools and of technology at non-Chapter 1 schools 

• Evaluations of new dropout prevention and dropout recovery programs 

• Evaluation of the effectiveness of tutorial assistance to students retaking the Exit-Level TAAS 

• Evaluation of the year-round education (YRE) concept implemented at Sanchez Elementary 

• identifying effective and ineffective programs with budget implications 

• Continued administration of a survey to former students to determine how well the District served them 
(a strategic objective) 

• Continued monitoring and reporting on the numbers of dropouts and at-risk students 

• Continued higher level data collection/analyses in the Chapter 1 and Chapter 1 Migrant evaluations 
mandated by state and federal reporting requirements 

• Evaluations of the Reading Recovery programs in AISD 

• Continued monitoring of the effectiveness of the Priority Schools 

• Continuing focus of the remaining local resources in providing technical assistance and responding to ad 
hoc requests for information from the School Board, the superintendent, central administration, schools, 
and agencies outside the District 
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OTHER ACTIVITIES 

Not all ORE activities are directly related to the production of a particular report or other publication. These 
activities are "invisible 1 ' to most observers of the District scene and must be acknowledged if the full picture of 
ORE is to be presented in the Agenda. 

Systemwide Testing and School Support is in some ways the most visible of the ORE components because every 
teacher, student, and parent of the District interacts with its products, yet the range of activities undertaken by 
the staff is probably poorly understood. In addition to coordinating the administration of the Iowa Tests of Basic 
Skills (TTBS) at grades 1 and 2, the Norm-referenced Assessment Program for Texas (NAPT) in grades 3-11, 
and the Texas Assessment of Academic Skills (TAAS) in grades 3-8 and Exit Level, Systemwide Testing and 
School Support eL jages in many other activities: 

• Scoring the ITBS and providing standard and requested analyses of all tests; 

• Coordinating state-required field testing of new TAAS tests; 

• Implementing advanced placement test procedures; 

• Keeping up with and providing information about the Preliminary Scholastic Aptitude Test (PSAT); Scholastic 
Aptitude Test (SAT), American College Testing program (ACT), and the Texas Academic Skills Program 
(TASP); 

• Tracking students who have not met mastery under the Texas Educational Assessment of Minimum Skills 
(TEAMS) program; 

• Coordinating National Assessment of Educational Progress (NAEP) testing; 

• Disseminating TAAS, NAPT, and ITBS objectives, measurement specifications, and test administration 
procedures; 

• Conducting a survey of former students; and 

• Coordinating a districtwide survey of administrative and professional staff. 

Systemwide Analysis and Development is that component of ORE in which tools for evaluation are developed, 
maintained, and improved. Longitudinal analyses also arise out of this component. These activities are not 
immediately apparent, but as their products begin to be employed in ORE evaluations and management studies, 
the systemwide impact is noticeable. The Report On School Effectiveness (ROSE), dropout tracking, and ORE's 
GENeric Evaluation SYStem, which began as developmental efforts, have had enormous effects on evaluation 
and on the District. More recently, the Implementation Index and new calculations of student mobility have 
shaped District decision making. 

The Systemwide Evaluation component of ORE is responsible, within the available resources, for evaluating any 
program, treatment, or initiative with priority for the District, whether locally or externally funded. Many 
externally funded evaluations contribute valuable information for the District far beyond their mandates. For 
example, the Chapter 1 and Chapter 2 evaluations devote considerable resources each spring to providing 
information necessary for the development of the project application for the following year. Tne Chapter 1 
evaluation has also borne a large part of the responsibility for monitoring the effectiveness of the Priority 
Schools. Systemwide Evaluation also contributes to the District by coordinating the process for screening 
requests by external researchers to conduct research in the District. 

All components of ORE engage in activities which are not reflected directly in evaluation reports. ORE staff are 
often called upon to assist with priority projects in the District, such as AISD's accountability system, and to 
furnish technical assistance to campuses, addressing evaluation requirements in grant applications, for example. 
Ad hoc requests for ORE assistance often arise from an information need of the Board of Trustees, the 
superintendent, or central administrators. Being able to respond to these requests quickly and accurately is one 
of the most important assets that ORE brings to the District. That ability is derived in part from the fact that the 
data collection of the mandated projects has allowed ORE to develop a computerized fund of information that can 
be drawn upon for many different uses. 



Other examples of the variety of ORE activities which do not lead directly to evaluation reports but contribute to 
District improvement can be found in the following list. 

• Improving the utility of achievement information provided to instructional staff 

• Coordinating dropout record keeping and at-risk identification to meet State reporting requirements 

• Contributing to the coordination of districtwide data collection and reporting by participation on the 
Information Services Committee (ISC) 

• Conducting needs assessments 

• Working with other AISD central offices, including Planning and Development, State and Federal 
Programs, and School Support Services, to assist in the acquisition of federal and state grants for 
special student programs such as Title II Math/Science, Title VII, and Drug-Free Schools 

• Working wi'l, the Evaluation Advisory Committee (EAC) composed of teachers, principals, university 
professors, and lay citizens charged with the oversight of ORE's activities 

• Providing information and technical assistance to projects and initiatives such as at-risk counselors, Project 
Mentor, Believe in Me, Pathways, Opportunities for Youth, and the School of the Future 

• Working with the area superintendents and principals in the four operations areas to improve 
communications and information flow between ORE and the schools 

• Making presentations about testing and evaluation results to parents, community groups, teachers, and 
administrators 

• Working with staff in the Department of Management Information to generate a School Profile 
containing a wide range of student, staff, and financial information for each AISD campus to be used in 
campus planning and improvement efforts 

• Evaluating, through Chapter 1, the Sanchez intersession tutoring program, Helping One Student to 
Succeed (HOSTS) labs, Higher Order Thinking Skills (HOTS) labs, Writing to Read labs, Content 
Mastery, and summer school programs 

• Evaluating the Austin Youth River Watch Program, a cooperative project between the Colorado River 
Watch Foundation (CRWF), Inc. and the City of Austin involving at-risk AISD students 

• Completing TEA studies mandated by the Legislature 

• Maintaining data files of students in major special programs 3d providing information to central and 
school staffs, as well as TEA 
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• Contributing to educational improvement through participation in or sharing information with statewide 
and national organizations which address issues that impact AISD (i.e., Joint Urban Evaluation 
Council, National Association of Test Directors, Directors of Research and Evaluation, Southwest 
Educational Research Association, American Educational Research Association, Annual Texas Testing 
Conference, etc.) 

• Sending requested copies of ORE reports and papers to interested persons and organizations worldwide 

• Responding to requests for information from AISD staff and others, both in the local community and 
across the country 

• Providing information for analysis by external, non-AISD researchers 

• Serving as a liaison to TEA to receive, interpret, and implement rules and regulations regarding student 
assessment 

• Compiling data for TEA's Prekindergarten Longitudinal Study 

• Assisting with the pilot in AISD of TEA's results-based monitoring (RBM) system for accreditation 

• Providing information in response to state reporting mandates such as HB 1758 (required posting of 
school district information) and the Public Education Information Management System (PEIMS) 

• Producing achievement profiles, detailing performance by grade, test, and ethnic group for each campus 

• Administering and scoring prekindergarten, kindergarten, and first-grade tests in English and Spanish, 
and conducting ongoing testing on Chapter 1 campuses for students without test scores 

• Providing evaluation information required by the State for the Pregnancy, Education, and Parenting 
(PEP) and Single Parents/Displaced Homemakers programs 

• Providing staff development assistance to campuses, departments, principal groups, and other school 
district staff 

• Providing review of TEA studies in the formative stages 

Another activity in which ORE will likely be involved in 1993-94 is the evaluation of the effect of changes 
brought about by waivers the District has received from state rules and regulations. In the absence of direct 
requests to ORE for evaluation from the instructional personnel who sought the waivers, ORE will provide 
assistance to the staff responsible for meeting state requirements for information; however, formal evaluations by 
ORE of the effect of waivers are not included in this Agenda. 

The 1993-94 District Improvement Plan specifies several major activities for which ORE is responsible, but a 
final determination on the implementation of some of these activities has not been made. Therefore, these 
activities are not included in the 1993-94 Agenda. Likewise, some activities associated with the implementation 
of AISD's Strategic Plan are not included in the Agenda because there will be a review and update of the 
Strategic Plan later this year. 

The 1993-94 Agenda matches the available evaluation resources with mandated or requested evaluation activities. 
If additional activities are identified during the year, some adjustments in planned activities would need to be 
made to allow resources to be directed to these new activities. ORE likely will be involved in activities 
associated with the review and update of AISD's Strategic Plan. 
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ABOUT ORE'S EVALUATION PLANS 



ORE'S evaluation plans are divided into four sections: 

1 . A heading , which gives the name of the program and the person(s) to contact about the evaluation; 

2. Program Description , which provides information about the purpose of the program, the components 
of the program, the activities of the program designed to accomplish the program's goals and 
objectives, the resources allocated to the program, and other details about the program's functioning 
relevant to understanding its workings; 

3. Evaluation Outl ine, which poses the decision or other (key or educational issues) questions and their 
related evaluation questions, along with the information sources to be accessed to answer the 
evaluation questions; and 

4. Information Needs (optional), which lists the information needs that are not addressed in the decision 
question section, which may include information required for annual TEA reports, applications, 
interim reports, etc. 

The decision and evaluation questions format, which has been used by ORE for many years, derives from the 
decision-making model of evaluation to which ORE subscribes. In this model, the major audience or reference 
group for evaluation results is decision makers. As ORE views evaluation, decision questions are to be 
answered by decision makers based on evaluation information provided to them; they are not answered by 
evaluation staff. Evaluation staff may at times make recommendations about a program, but decisions about a 
program need to be made by program staff, the School Board, and others. The evaluation questions which are 
enumerated under the decision questions are intended to guide the evaluation toward collecting, analyzing, and 
reporting information which will enable decisionmakers to answer the decision questions. Put another way, 
decision questions are an organizing principle around which evaluation questions are grouped. They are not 
intended to be taken literally as questions to be answered in an evaluation report. The "answer" to a decision 
question is a program or District decision, which lies outside the evaluation. 

Often, though not always, decision questions are phrased in terms of whether a program should continue as is, 
be modified, or be discontinued. Although this wording sometimes has raised the ire of peisons with an 
investment in a program's continuance, all special programs are subject to this decision whether the question is 
made explicit or not. 

Recognizing that decision questions are often unvarying (and perhaps too formulaic), ORE developed a variation 
on the decision question format that permits writers of evaluation plans to organize evaluation questions around 
key issues or educational issues. These types of questions help to focus the evaluation on the issues which gave 
rise to the program originally. Like decision questions, however, these questions are not intended to be 
answered by the evaluation staff; answers to the evaluation questions associated with them will enable 
decisionmakers to address the issues. 
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In 1993-94, acknowledging that many of its activities fall outside the scope of evaluation proper and more in the 
realm of iranagement studies and needing to reflect these activities in a more tractable format, ORE developed 
an alternative to the evaluation plan termed the management plan. The management plan shares the format of 
ORE's evaluation plans but differs in its emphasis and terminology. Like the evaluation plan, the management 
plan is structured around decision questions to be answered by decisionmakers (or key issues to be addressed), 
with subsidiary questions to be answered by evaluation staff (here termed management questions); the 
information sources to be utilized are listed; the date by which the information is needed is given, and contact 
persons are designated. Unlike evaluation plans, management plans are concerned with information required for 
management decisions and do not have reference to a special program. The decisions to be made by 
decisionmakers for whom management plans are written concern the efficient functioning of the District rather 
than whether a particular program should be continued as is, modified, or discontinued. Management questions 
are intended to yield information for decision makers so that they can make management decisions, about 
whether to add or reallocate resources or to address a situation in a different fashion, for example. Management 
questions often take the form of counting or inventory questions, such as, M How many dropouts are there?" or 
H How many AISD high school graduates are employed after graduation?" Answers to these types of questions 
will enable decisionmakers to address the underlying issues, e.g., by recommending changes in the curriculum. 

Instead of a Program Description, the management plan has a section entitled Management Context. As in the 
completed staff work (CSW) format utilized by ORE or the format for School Board agenda items, this section 
of the management plan contains background information and addresses administrative considerations. 
Management Context describes the context out of which the need for the management information arose, the 
purpose for which the information is intended, the duration for which the information is to be collected, and any 
qualifications or limitations on the information which should be understood. 

Readers of this Agenda are encouraged to direct their comments and questions about evaluation in general and 
about the individual evaluation and management plans which follow to the ORE contact persons named in the 
plans. 
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CHAPTER 1 



EVALUATION PLAN 



Contact Person: 
Shirin Catterson, Ph J>. 



PROGRAM DESCRIPTION 

Chapter 1 is a continuing program supported by 
funds from the Department of Education under the 
Elementary ad Secondary Education Act (ESEA). 
The purpose of Chapter 1 is to provide for the 
learning needs of educationally disadvantaged 
students in school attendance areas with high 
concentrations of children from low-income 
families. Chapter 1 provides supplemental 
assistance to the regular school program. AISD's 
funding from Chapter 1 for 1993-94 is $9,209,187. 

Participation of schools in AISD's Chapter 1 
Program is determined by economic need. Schools 
with a higher concentration of low-income families 
than the District average are eligible to receive 
Chapter 1 services. These schools are then ranked 
by the percent of low-income students who reside 
in their attendance areas. Schools are chosen for 
Chapter 1 services based on their low-income 
rankings and the District's Chapter 1 funds. 
Standardized test scores are used to determine how 
many students may be served at each school. 
Students with the lowest test scores have priority 
for service. A description of different components 
of Chapter 1 follows. 

Full-Day Prekindergarten 
The State funds half-day prekindergarten for all 
low-income and limited-English-proficient (LEP) 
students. In 28 of the 31 Chapter 1 schools, 
Chapter 1 is funding an additional half-day session 
for Chapter 1 eligible students providing diem with 
a full-day prekindergarten experience. 

Chapter 1 Supplementary Instruction 

The main objective of this instructional component 
is to improve Chapter 1 students* reading skills. 
This component is supplementary to and coordi- 
nated with the District's basic reading program. Its 
primary purpose is to provide additional assistance 
to students deficient in language and/or reading 
skills. During the 1993-94 school year, 31 schools 
are providing students with a variety of supplemen- 
tary services. These services include: supplemen- 
tary reading instructors, Reading Recovery, 
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Program 
Description 
Continued 



Content Mastery labs, Higher Order Thinking 
Skills (HOTS), Computer Curriculum Corpora- 
tion labs, Jostens Learning labs, Writing to Read, 
Writing to Write, Help One Student to Succeed 
(HOSTS), and SABES. 

Schoolwide Projects (SWP) 
All students at a SWP are eligible to receive 
Chapter 1 services, regardless of their test scores. 
To be a schoolwide project, 75% or more of the 
students in the school's attendance area must be 
from low-income families. Chapter 1 funds can 
then be used to carry out a project to upgrade the 
educational program in the entire school, such as 
lowering the pupil-teacher ratio, or purchasing 
computers or computer software. It is important 
to note that although SWPs can serve all students, 
their evaluations are based on the achievement 
gains of those students who began the school year 
with standardized test scores below the 31st 
percentile. Chapter 1 is funding 26 schoolwide 
projects for the 1993-94 school year. 

Parental Involvement 

Chapter 1 staff and parents consult with each 
other through Chapter i Districtwide Parental 
Advisory Council (PAC) meetings several times 
during the school year. In this manner, parents 
can advise the District in the planning and the 
operation of the programs, as well as receive 
up-to-date information and training on areas of 
interest, such as helping their children with 
reading at home. 

Nonpublic Schools 

Computer-assisted instruction in reading and 
mathematics is provided to one nonpublic school. 
This school provides Chapter 1 services to 
low-achieving students who reside in the 
attendance areas of Chapter 1 public schools. 

Institutions for the Neglected or Delinquent 
Chapter 1 funds are provided to nine institutions 
for neglected or delinquent students to fund 
support services for students with emotional, 
psychological, or behavioral problems. 
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Decision 
Question 
1 



Should the Full-Day Prekindergarten Component be continued as it is 
or be modified? 



Date Needed: June 1994 

Evaluation Questions 1-1 through 1-8 will be addressed by Chapter 1 evaluation. 

EVALUATION QUESTIONS INFORMATION SOURCES 

1-1. How many students were served by the * Attendance File (ATND) (Ongoing) 

regular, bilingual, and ESL pre-K • Student Master File (STUD) (Ongoing) 

classes? By age, sex, ethnicity, and 
schools? By full-day and half-day 
classes? 



1-2. What was the attendance rate for pre-K • ATND File (Ongoing) 

students? Did this vary by type of 
class? By full day/half day? 



1-3. How did the PPVT-R pre- to posttest 
gains made by pre-K students 
compare: 

• To the national norm? 

• To previous years? 



• Peabody Picture Vocabulary Test-Revised 
(PPVT-R) File (Sept./Oct. 1993, April 
1994) 



1-4. How did the PPVT-R pre- to posttest • PPVT-R File (Sept./Oct., April) 

gains compare: 

• In the ftill-day and half-day 
classes? 

• In the bilingual and English 
classes? 

• Across different types of 
classes tor students with 
the lowest pretest scores? 
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Decision 
Question 1 
Continued 



EVALUATION QUESTIONS 

1-5. For Spanish LEP students who took the 
Spanish TVIP and English PPVT-R, 
how did pre- to posttest gains compare? 



1-6. Was there any difference between the 
achievement gains of full-day and half- 
day bilingual students? 



1-7. Were there differences in the achieve- 
ment gains of pre-K students at different 
campuses? 



1-8. What were the strengths and the areas in 
need of improvement in the implemen- 
tation of the pre-K component? 



Evaluation Question 1-9 will be addressed by the Chapter 2 evaluation. 



1-9. What was the Chapter 2 Formula cost • Program Record (May 1994) 

per student? How cost-effective was the 
program? 



INFORMATION SOURCES 

• Test de Vocabulario en Imagenes Peabody 
(TVIP) File (Sept./Oct., April) 

• PPVT-R File (Sept./Oct., April) 

• Language (LANG) File (Ongoing) 



TVIP/PPVT-R File (Sept./Oct./April) 
LANG File (Ongoing) 



PPVT-R/TVIP File (Sept./Oct., April) 



• STUD File 



Coordinator Interview (Spring 1994) 
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Decision 
Question 
2 



Should AISD change the structures and approaches traditionally used 
in Chapter 1? 



Date Needed: June 1994 
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EVALUATION QUESTIONS 

2-1. What were the achievement gains of 
students who received Supplementary 
Reading instruction? How did their 
gains compare with those of similar 
students who did not receive Chapter 1 
Supplementary Services? 

2-2. Were there any differences in 
effectiveness among the different 
supplementary programs? 

2-3. How did the TAAS achievement of 
Chapter 1 -served students in 1993-94 
compare with the achievement of 
students with similar spring or fall 

1993 scores who were not served by 
Chapter 1? 

2-4. What percentage of Chapter 1 students 
became ineligible for the 1994-95 
Chapter 1 Program based on their spring 

1994 ITBS or NAPT scores? How did 
this compare to last year? 

2-5. What was the average number of years 
of Chapter 1 services received by the 
1993-94 Chapter 1 students (by grade)? 



INFORMATION SOURCES 

• Normed-referenced Assessment Program for 
Texas (NAPT) File (April) 

• Iowa Tests of Basic Skills (ITBS) File 
(April 1994) 



NAPT File (April) 
ITBS File (April) 

Record of Student Service (ROSS) Forms 
(Ongoing) 

Texas Assessment of Academic Skills 
(TAAS) File (May) 
ROSS Forms (Ongoing) 
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ROSS Forms (Ongoing) 
ITBS File (April) 
NAPT File (April) 
Chapter 1 File (Ongoing) 



Chapter 1 File (Ongoing) 
ITBS File (April) 
NAPT File (April) 
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Decision 
Question 2 
Continued 



EVALUATION QUESTIONS 



INFORMATION SOURCES 



2-6. Were there by-campus differences in the 
achievement gains of Chapter 1 
students? 



ROSS Forms (Ongoing) 
ITBS File (April) 
NAPT File (April) 
TAAS File 



2-7. How do served Chapter 1 students do on 
the higher order thinking skills items on 
the NAPT? How does this compare 
with other students with similar spring 
or fall 1993 scores who were not served 
by Chapter 1? 

2-8. How successful was the implementation 
of the Chapter 1/Chapter 1 Migrant 
instructional programs? 

• What concerns/strengths were 
identified by Chapter 1/Chapter 1 
Migrant administrative staff? 

2-9. What is the average cost of the 

program per estimated student contact 
hour? 



NAPT File (April) 
ROSS Forms (Ongoing) 



Administrative Staff Interview 
(Spring) 



ROSS Forms (Ongoing) 

Chapter 1 Application for Funding (August) 



2-10. How successful have the Chapter 1 
programs been in improving students' 
achievement levels longitudinally? 



ITBS File (April) 
NAPT File (April) 

Longitudinal Chapter 1 File (Ongoing) 
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Decision 
Question 
3 



Should Chapter 1 change the way students and schools are selected? 



Date Needed: June 1994 
EVALUATION QUESTIONS INFORMATION SOURCES 



3-1. What procedures are in place to identify 
Chapter 1 students for service 
eligibility? 



Testing for Chapter 1 Eligibility Manual 
(Sept.) 



i 
i 
i 
i 
i 
i 
i 
i 
i 
i 
i 
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3-2. How many students were served at each 
grade level by sex, ethnicity, and type 
of supplementary program? 



3-3. How many students were special tested? 
Why? Did students tested differ by 
grade, ethnicity, or sex? 



3-4. What percentage of Chapter 1 -served 
students are low-income? 



3-5. What number and percentage of students 
eligible for Chapter 1 services received 
supplementary instruction from another 
source? 



ROSS Forms (Ongoing) 
STUD File 



• Special Test File (Ongoing) 

• STUD File 



ROSS Forms (Ongoing) 
Cafeteria File (January) 



• STUD File 
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Decision 
Question 3 
Continued 



EVALUATION QUESTIONS 

3-6. What percentage of eligible students are 
served by Chapter 1? What percentage 
of eligible LEP students are served by 
Chapter 1? How does this compare 
with last year's figure? Is there by- 
campus variation in the percentage of 
eligible students served? 



INFORMATION SOURCES 

• ROSS Forms (Ongoing) 

• LANG File (Spring) 



3-7. What percentage of grade K students are 
eligible for Chapter 1 by school? 



Boehm Test of Basic Concepts-Revised File 
(Nov.) 



3-8. What percentage of grade 1 students are 
eligible for Chapter 1? By school? 



Metropolitan Readiness Tests File (Sept.) 
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Decision i How succ essfiil were the original 16 Priority Schools after their 
Question, | seventh year? 

4 



Date Needed: June 1994 
EVALUATION QUESTIONS INFORMATION SOURCES 

4-1. How did the teacher transfer request rate • EMR File (Ongoing) 

for the Priority Schools compare with 
the transfer request rate in the rest of 
the District? Has this changed over the 
years? 



4-2. How do the standardized test scores of 
students at each priority school compare 
with those of other AISD students and 
the national averages? 

4-3. What effect did lowering the pupil- 
teacher ratio have on students' 
achievement gains at each school? 



ITBS File (April) 
NAPT File (April) 



ITBS File (April) 
NAPT File (April) 
ATND File (End of Year) 



4-4. How do the achievement gains made by * ITBS File (April) 

low achievers in the lowered PTR • NAPT File (April) 

compare to low achievers served in the • ATND File (End of Year) 

Chapter 1 supplementary program? • ROSS Forms 
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Decision 
Question 4 
Continued 



EVALUATION QUESTIONS INFORMATION SOURCES 

4-5. What were the TAAS passing and • TAAS File (March, May) 

mastery rates in each school on: 

• TAAS-Writing (March, Grade 4) 

• TAAS-Reading & Mathematics (May, 
Grades 3-6) 

• TAAS-Science & Social Studies 
(May, Grade 4) 

4-6. What was the student attendance rate in • ATND File (Ongoing) 

each school? How did this compare 
with the District and with previous 
years? 

4-7. What were the promotion/retention/ • STUD File (June) 

placement rates for each of the Priority 
Schools? How does this compare with 
other AISD elementary schools? 

4-8. What Pupil/Teacher/Ratio (PTR) was • ATND File (June) 

achieved at each grade level at each 
campus? Did this match prescribed 
levels? 



4-9. What activities occurred at each campus 
to involve parents and community 
members? 

4-10. What is the achievement level of 
students who have attended priority 
schools for one or more years since 
1987? How do they compare with other 
students in the District? 

4-11. What was the achievement gain of 
students who had spring or fall 
1993 test scores of below the 31st 
percentile? How do these gains 
compare with the gains in 1991- 
92 and 1992-93? 



• Parent Training Specialist Records Review 
(Spring) 



• ITBS File (April) 

• NAPT File (April) 

• TAAS File (May) 

• STUD File 



• ITBS File (April) 

• NAPT File (April) 

• MRT File (Sept.) 
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How effective were the Supplementary programs implemented with 
Chapter 1 funding? 



Date Needed: June 1994 
EVALUATION QUESTIONS INFORMATION SOURCES 



5-1. Were the summer school programs held • Summer School Data (Aug.) 

in July and August effective? 

5-2. How effective were the following • GENeric Evaluation SYStem (GENE3YS) 

programs at Chapter 1 Schools? (Summer 1994) 

• Extended Day? • ROSS Forms 

• Help One Student to Succeed 

(HOSTS)? 

• Content Mastery? 

• Intersession Tutoring Program 

(Sanchez)? 

• Higher Order Thinking Skills 

(HOTS)? 

• Writing to Read? 

• Writing to Write? 

• SABES/SESOS? 
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How effective was the Reading Recovery Program in AISD? 



Date Needed: June 1994 
Evaluation Questions 6-1 through 6-9 will be addressed by the Chapter 1 evaluation. 



EVALUATION QUESTIONS 

6-1. How many grade one students were 
served in the second year of 
implementation? What was the 
average length of stay? 



INFORMATION SOURCES 
• ROSS Forms 



6-2. What percentage of children served by 
the Reading Recovery program 
districtwide were successfully exited 
from the program? 



ROSS Forms 



6-3. How did successfully exited Reading • ITBS File (April) 

Recovery students compare with other • MRT File (Fall 1993) 

first graders served by Chapter 1 on a 
nationally normed standardized test? 



6-4. What was the rate of promotion for 
students who were exited from the 
Reading Recovery program compared 
with the District promotion rate? 



• STUD File (Promotion/Retention/Placed) 

* ROSS Forms 
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Decision 
Question 6 
Continued 



EVALUATION QUESTIONS 

6-5. What responses to the program were 
made by teachers, administrators, 
Teacher Leaders, parents, and children? 



6-6. How cost-effective is the Reading 

Recovery program compared to other 
supplementary reading programs? 



6-7. What is the retention rate for trained 
teachers (from 1992-93 to 1993-94)? 
How does this compare to other urban 
districts? 



6-8. Are the effects of Reading Recovery 
sustained in the second grade for 
students who were exited from Reading 
Recovery in the first grade? 



6-9. Has teacher training been sufficient for 
proper implementation of the program? 



Evaluation Questions 6-10 through 6-12 



6-10. How was the training applied during 
sessions with the Reading Recovery 
students? With regular classroom? 

6-11. Would this program continue if federal 
funds were not available? 

6-12. What was the Chapter 2 Formula cost 
per participant? 



INFORMATION SOURCES 

• Surveys of Reading Recovery Teachers and 
Classroom teachers 

• Interviews with Teacher Leaders and 
Principals 



• STUD File 

• Finance Office 

• ITBS File (April) 

• Employee Master Record (EMR) File 

• AISD Teacher leaders 

• U.S. Reading Recovery Headquarters at 
Ohio State University 



• ITBS Reading Comprehension 

• Rank Order of Students According to 
Reading Skills by Second Grade Teachers 



• Teacher Surveys 

• Teacher Leader Interviews 



ill be addressed by the Chapter 2 evaluation. 



• Classroom Observations (Spring 1994) 



• Program Coordinator Interview (February 
1994) 

• Program Records (May 1994) 
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Decision 
Question 
7 



How effective were Integrated Learning Systems (Computer 
Curriculum Corporation (CCC) and Jostens Learning) at Chapter 1 
Schools? 



Date Needed: June 1994 

Please refer to the evaluation plan for Technology in Instruction (pages U-l to U-4). This evaluation 
will be coordinated by Chapter 1 utilizing both Chapter 1 and local Office of Research and Evaluation 
resources. 
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Decision 
Question 
8 



Should the Parental Involvement Component be modified? If so, how? 



Date Needed: 
EVALUATION QUESTIONS 
Were the component's objectives met? 

How many Chapter 1 Districtwide PAC 
meetings and training sessions were held 
during the 1993-94 school year? 



June 1994 

INFORMATION SOURCES 

• PAC Records (Ongoing) 

• Administrative Staff Interview (Spring) 

• PAC Records (Ongoing) 



Did more Chapter 1 parents attend • PAC Records (Ongoing) 

Districtwide PAC meetings during 
1993-94 than during 1992-93? 



8-4. How successful was the imple- • Administrative Staff Interview (Spring) 

mentation of the Parental 
Involvement Component? What 
concerns/strengths were identified 
by Chapter 1 staff? 



i 
i 
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Should the Chapter 1 Nonpublic Schools Component be modified? If 
so, how? 



Date Needed: June 1994 



EVALUATION QUESTIONS 
9-1. Were the component's objectives met? 



9-2. How many students were served by the 
Chapter 1 nonpublic schools by age, 
grade, gender, and ethnicity? 



INFORMATION SOURCES 

• Chapter 1 Service Report for Nonpublic 
Schools (Spring) 

• Chapter 1 Service Report for Nonpublic 
Schools (Spring) 



9-3. What evidence is there that students at 
nonpublic schools receiving Chapter 1 
services were successful? 



Chapter 1 Service Report for Nonpublic 
Schools (Spring) 



9-4. How successful was the implementation 
of this component? What concerns/ 
strengths were identified by Chapter 1 
staff? 



Administrative Staff Interview (Spring) 
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Should the Chapter 1 Component for Institutions for Neglected or 
Delinquent (N or D) youth be modified? If so, how? 



Date Needed: June 1994 



EVALUATION QUESTIONS 
10-1. Were the component's objectives met? 



INFORMATION SOURCES 

• Chapter 1 N or D Service Report (Spring) 



10-2. How many students were served by the 
Chapter 1 N or Ds institutions-by age, 
sex, grade, and ethnicity? 



Chapter 1 N or D Service Report (Spring) 



10-3. How many of the N or D students • Chapter 1 N or D Service Report (Spring) 

served by Chapter 1 were also 
identified as LEP and/or special 
education students? 



10-4. What are the goals and objectives of • Interviews with the Administrative Staff of 

Chapter 1 services at these institutions? N or Ds (Fall, Spring) 

• How successful were these 
institutions in achieving their 
goals? 

• How successful were the served 
N or D students? 



10-5. How successful was the implementation • Administrative Staff Interview (Spring) 

of this component? 
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Decision 
Question 
11 



Were programs funded by Chapter 1 effective relative to their costs? 



Date Needed: June 1994 

EVALUATION QUESTIONS INFORMATION SOURCES 

11-1. What were the costs for each of the • Chapter 1 Application (June) 

programs/activities funded by • Program Records (June) 

Chapter 1? 



11-2. Were the programs funded by Chapter 1 • Cost Effectiveness Comparisons (June) 

cost effective? 
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Information 
Needs 



Needs Assessment for the 1994-95 Chapter 1 Application 



Date Needed: June 1994 
INFORMATION QUESTIONS INFORMATION SOURCES 



What percentage of the students residing in 
each AISD attendance area are from low- 
income families? 



• Student Master File (Ongoing) 

• Cafeteria File (Ongoing) 



How many students in each school scored 
below selected percentile points on the 
ITBS and NAPT? 



• Student Master File (Ongoing) 

• ITBS File (April) 

• NAPT File (April) 



3. How many students would be eligible for 
Chapter 1 services based on various 
combinations of criteria for campus and 
student eligibility? 



ITBS File (April) 
NAPT File (April) 
MRT File (September) 
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Information 
Needs 



Annual Program Documentation for the Texas Education Agency 



Date Needed: June 1994 
INFORMATION QUESTIONS INFORMATION SOURCES 



How many students were served (by grade, 
sex, ethnicity, and age) by Chapter I, 
including service at nonpublic schools and 
at N or Ds? 



• ROSS Forms (ongoing) 

• Chapter 1 Service Report for Nonpublic 
Schools (Spring) 

• Chapter 1 N or D Service Report (Spring) 



What were the achievement gains for 
students served by Chapter 1 during 1993- 
94 in normal curve equivalents (NCEs) by 
grade and by campus? 



What percent of students served by 
Chapter 1 were retained at each school? 

Which campuses met their "Preponderance 
of Evidence 1 * requirements? 

• A five point average gain on the 
PPVT-R for pre-K students? 

• A five percentile point increase 
on the Boehm-R for kindergarten 
students served by Chapter 1? 



• ROSS Forms (Ongoing) 

• ITBS File (April) 

• NAPT File (April) 

• Chapter 1 Service Report for Nonpublic 
Schools (Spring) 

• Student Master File (June) 



• PPVT-R File (Sept.,Oct.,April,May) 

• Boehm-R File (Nov., May) 

• ITBS File (April) 

• ROSS Forms (Ongoing) 

• Student Master File (June) 



• An average grade equivalent 
score of 1.6 on the ITBS Reading 
Comprehension for Chapter 1- 
served grade 1 students? 
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Information 

Needs 
Continued 



EVALUATION QUESTIONS 



INFORMATION SOURCES 



5. 



• 98% promotion/placement rate for 
Chapter 1-served students? 

• A 5% increase in percent of 
Chapter 1-served students who 
passed the TAAS Reading test? 

What was each campus' progress on 
meeting their NCE gain requirements? 
Which schools are 01 , off, or continuing 
on program improvement? 



ITBS File (April) 
NAPT File (April) 
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CHAPTER 1 
MIGRANT 



EVALUATION PLAN 



Contact Persons: 
Shirin Catterson, PhJD. 
Wanda Washington 



PROGRAM DESCRIPTION 

The Chapter 1 Migrant Program is a federally 
funded project designed to meet the unique needs of 
the District's migrant students. Although priority is 
given to currently migrant students, both currently 
migratory and formerly migratory children may be 
served by the Migrant Program. A currently 
migratory child is one (a) whose parent or guardian 
is a migratory agricultural worker or migratory 
fisher; and (b) who has moved within the past 12 
months from one school district to another to enable 
the child, the child's guardian, or a member of the 
child's immediate family to obtain temporary or 
seasonal employment in an agricultural or fishing 
activity. Students who remain in the District 
following their year of current eligibility are 
considered formerly migratory students for a period 
of five years. For 1993-94 the funding level 
decreased to $198,743, due to a reduced enrollment 
of Migrant Students in the District. 

The activities of the Migrant Program are centered 
around: 

• Recruitment of students, parental 
involvement, and 

• A supplementary instructional program 
for first grade through high school 
students. 

Recruitment and Parental Involvement 

In order to be eligible for the services provided by 
the Migrant Program, the parents (or the guardians) 
of the student have to complete a Certificate of 
Eligibility/Identification. Home visits to parents are 
made throughout the year as new migrant students 
are located and identified. When the Eligibility/ 
Identification forms are completed, they are sent by 
the Migrant Student Record Transfer System 
(MSRTS) Clerk to the Region XIII Education 
Service Center for entry into the National MSRTS 
data bank in Little Rock, Arkansas. 

The Chapter 1 Migrant legislation requires that staff 
consult parents in planning, operating, and 
evaluating the program. The Migrant legislation 
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also requires a Districtwide Parental Advisory 
Council (PAC). In this manner parents can advise 
the District in its planning and operation of the 
program, as well as receive up-to-date information 
and training on areas of interest-helping children 
with reading and mathematics at home, etc. 

Grades 1-6 

The instructional emphasis at these grade levels is 
supplementary support services in coordination 
with the regular school curriculum. 

Gra d es 7-12 

The instructional emphasis at these grade levels is 
tutorial services coordinated with the regular school 
curriculum. 



Program 
Description 
Continued 
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Decision 
Question 
1 



Should the Chapter 1 Migrant Supplementary Tutorial Component be 
modified? If so, how? 



Date Needed: 
EVALUATION QUESTIONS 

1-1. Were the achievement objectives met 
at: 

• Grades 1-6? 

• Grades 7-8? 

• Grades 9-12? 



June 1994 

INFORMATION SOURCES 

Norm-referenced Assessment Program for 
Texas (NAPT) File (April 1994) 
• Iowa Tests of Basic Skills (TTBS) File 
(April 1994) 



1-2. How do the gains/achievement scores * NAPT File (April 1994) 

made this year by Migrant students in * ITBS File (April 1994) 

grades 1-12 compare with the gains/ 
achievement scores of Migrant students 
in 1992-93? 



1-3. How successful was the implemen- 
tation of the Supplementary Tutorial 
Component? 

• What concerns/strengths were 
identified by Chapter 1 Migrant 
staff? 



Administrative Staff Interview (Spring 
1994) 



1-4. How many students did Migrant 
teachers and tutors serve in each 
grade? 



Record of Student Service (ROSS) Forms 
(Ongoing) 

Program Records (Ongoing) 



1-5. What percentage of the Migrant • ROSS Forms (Ongoing) 

students served by a Migrant teacher 
were served by each instructional 
method (lab, team teaching, tutoring 
special class, and other)? How does 
this compare with 1992-93? 
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Decision 
Question J 
Continued 



EVALUATION QUESTIONS 

1-6. What number and percentage of 
migrant students received supple- 
mentary instruction from another 
source? 



INFORMATION SOURCES 
Migrant Student Master File 



1-7. What is the average cost of the 

program per estimated student contact 
hour? 



ROSS Forms (Ongoing) 

Migrant Application for Funding (August) 

Funding (August) 



1-8. How many students were served by the 
Migrant Program by age, sex, grade, 
and ethnicity? 



ROSS Forms (Ongoing) 
Tutors* Time Sheets 



1-9. How were the Migrant instructional 
funds spent? 

1-10. How many migrant students attended 
summer programs? 



Administrative Staff Interview (Spring) 



Migrant Records Review (Summer) 



1-11. What evidence of success was there for 
summer school participation? 



Summer School Records (Summer) 



1-12. Did the students who were served by 
the Migrant teachers make any gains 
on the Academic Excellence 
Indicators? Did they meet the goals 
set in the application? 



Academic Excellence Indicator System 
(AEIS) (June) 



1-13. How successful was the computer lab? 



Administrative Staff Interviews (Spring) 
Attendance File (May) 
Other Data Sources (May) 
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Decision 
Question 
2 



Should the Parental Involvement Component be modified? If so, how? 



Date Needed: June 1994 



EVALUATION QUESTIONS 

2-1. Were the component's objectives 
met? / 

2-2. How many Migrant Districtwide 

PAC meetings and training sessions 
were held during the 1993-94 
school year? 



INFORMATION SOURCES 

• , PAC Records (Ongoing) 

• Administrative Staff Interview (Spring) 

• PAC Records (Ongoing) 



2-3. Did more migrant parents attend • PAC Records (Ongoing) 

Districtwide PA " meetings during 
1993-94 than during 1992-93? 



2-4. How successful was the implemen- • Administrative Staff Interview (Spring) 

tation of the Parental Involvement 
Component? 

• What concerns/strengths were 
identified by program staff? 
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Decision 
Question 
3 



Should the Migrant Student Record Transfer System (MSRTS) 
Component be modified? If so, how? 



Date Needed: June 1994 



3-1. 



EVALUATION QUESTIONS 
Were the component's objectives met? 



3-2. What services did migrant students 
receive? 

3-3. How many migrant students (by grade, 
sex, and ethnicity) received some 
health services provided by the 
Migrant Program? 

3-4. Was the District's procedure for the 
processing of the migrant students' 
records efficient? 



INFORMATION SOURCES 

• MSRTS Records Review (Ongoing)) 

• MSRTS Clerk Interview (Spring) 
Migrant Student Master File (Ongoing) 

• MSRTS Clerk Interview (Spring) 



MSRTS Clerk Interview (Spring) 
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Needs Assessment for 1994-95 Chapter 1 Migrant Application 



Date Needed: June 1994 
INFORMATION QUESTIONS INFORMATION SOURCES 

1. What is the projected number of • Migrant Student Master File (Ongoing) 
migrant students enrolled in each 

• school (by grade) in the 1994-95 
academic year? 

2. What is the achievement level of • Migrant Student Master File (Ongoing) 
migrant students by school and by • ITBS File (April) 

grade? • NAPT File (April) 

3. What compensatory programs served • Migrant Student Master File 
migrant students at each grade for each 

school? How many migrant students 
were served by each? 

4. What are appropriate goals to set for • AEIS 
the served migrant students? 



Information 
Needs 
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TEA Report - 1993-94 School Year 



Date Needed: June 1994 



INFORMATION QUESTIONS 

1 . What is the total number of eligible 
migrant students? 

Regular term? 
Summer term? 

2. How many migrant seniors graduated? 

• Regular term? 
Summer term? 

3. How many migrant students in grades 2- 
12 who were served by a Migrant teacher 
or tutor had pretest scores in 1992-93 
and posttest scores in the 1993-94 school 
year? 

4. For the students in Question 3, what was 
the average NCE gain or loss by grade 
level for 1993-94? 

5. For migrant students with matched pre- 
and post-test scores on ITBS or NAPT 
and who were not served by a Migrant 
teacher, what was the average NCE gain 
or loss by grade level for 1993-94? 

6. Were the "Desired Outcomes" criteria set 
for the Migrant program met in 1993-94? 

• A three percent increase in Migrant 
student promotion rate from 87% to 
90%? 

• A 10% increase in the number of 
Migrant students taking a College 
Entrance Exam (from 60% to 70%)? 

• A 33% increase in students passing 
all sections of the TAAS (from 
17% to 5%)? 



INFORMATION SOURCES 

• Migrant Student Master File (Ongoing) 



Student Master File (Ongoing) 
Migrant Student Master File (Ongoing) 



ITBS File (April) 
NAPT File (April) 



ITBS File (April) 
NAPT File (April) 



ITBS File (April) 
NAPT File (April) 



Migrant Student Records 
College Entrance Exams 
TAAS File (March, May) 
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DISTRIBUTION 

OF I Chapter 1 /Chapter 1 Migrant 

REPORTS i School Staff Report 



Report 


Number of Copies and Recipient 


Date 


Chapter 1 /Chapter 1 Migrant 
Roster and ROSS Forms 


All Chapter 1 and Chapter 1 
Migrant teachers 


August and 
every six weeks 


Chapter 1 /Chapter 1 Migrant Final 
Reports 


All Chapter 1 and Chapter 1 
Migrant teachers 


August 


Pre-K Test Results 


Pre-K teachers 
Pre-K principals 


November and 
May 


MRT Results 


First grade teachers 
First grade principals 


October 


Boehm-R Results 


Kindergarten teachers 
Kindergarten principals 


November and 
May 


Other Analyses 


Program staff and others 


Upon request 


Observation Feedback 


Teacher and Computer Lab 
staff 


As needed 


NCE Gains 


Chapter 1 principals 


August 
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CHAPTER 2 
FORMULA 



EVALUATION PLAN 



Contact Person: 
Julia Griffith 



PROGRAM DESCRIPTION 

Chapter 2 Formula provides federal funds to 
states through the Elementary and Secondary 
Education Act (ESEA) of 1965 as amended 
1988 by Public Law 100-297, States earn 
Chapter 2 Formula funds based on their school- 
age population and then allocate at least 80% of 
these fUnds to local districts based on enrollment. 

Chapter 2 Formula can support programs or 
services in several target assistance areas. The 
Austin ISD programs receiving Chapter 2 ^ 
Formula funding in 1993-94, and the assistance 
areas targeted, are listed below. 

At-Risk Students 

• Student Retreat Program 

($21,886 Chapter 2; $113.01 in additional 
funding from Drug-Free Schools) 

• MegaSkills ($36,656 Chapter 2; $16,400 
in additional funding from Chapter 1) 

Instructional Materials 

• Library Resources ($63,425 Chapter 2) 

• Supplementary Materials (Multisensory 
Teaching Approach {MTA}, Project Read) 
($50,000 Chapter 2) 

Staff Development 

• Spanish Academy ($40,136 Chapter 2) 

• Staff Development ($24, 1 15 Chapter 2) 

Training in Identifying Students At-Risk For 
Illiteracy 

• Reading Recovery Teacher Leader 
($63,360 Chapter 2) 

• Reading Recovery Teacher Leader In 
Training and Campus Locations 
($63,185 Chapter 2) 

Personal Excellence 

• Academic Decathlon ($42,886 Chapter 2) 
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Innovative Projects 

• Full-Day Prekindergarten ($82,310 
Chapter 2) 

• Middle School "Transition In" Program 
($15,039 Chapter 2) 

No programs were funded in the assistance area 
of innovative programs. 

Chapter 2 Formula funds are also available 
through AISD to nonpublic, nonprofit schools 
located within AISD boundaries. Schools that 
meet the eligibility requirements are allocated 

funds on a per-pupil basis for the purchase of 

■ 1 — items approved by the Texas Education Agency. 

Funds for the 1993-94 school year have been 
Program allocated for library materials, computer software 

Description hardware, instructional materials, and staff 

Continued development. 

In 1993-94, AISD will receive $535,752 through 

Chapter 2 Formula (in addition to $42,261 rolled 

— forward from 1992-93 and $53,891 not allocated 

during 1992-93 for a total of $631,904). 
Included in the total allocation are $10,258 for 
indirect costs and funds for the evaluation and 
management of Chapter 2 Formula. Evaluation 
will be provided through a full-time Chapter 2 
Formula evaluation associate. 
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Educational 
Issue 
1 



ACADEMIC DECATHLON: Is this program contributing to 
academic excellence at the secondary level in AISD? Should the 
program be continued in its present form? 



Date Needed: June 1994 



EVALUATION QUESTIONS INFORMATION SOURCES 

1-1. How was the Academic Decathlon • Program Coordinator 

implemented in 1993-94? Which schools . Interview (April 1994) 

were active in the program? How many 
students participated from each school? 
How many staff were involved? 



1-2. How many students were recruited, 

practiced, and participated in the regional 
meet? What were the characteristics of 
the students involved (grade, sex, 
ethnicity)? 



• Program Records (Spring 1994) 

• Student Master File (Spring 1994) 



1-3. According to Academic Decathlon 
coaches, were recruitment efforts 
successful? 

1-4. How effective was the Academic 
Decathlon in promoting academic 
excellence? 



• Academic Decathlon Coaches Survey (April 
1994) 



• Program Coordinator Interview 
(April 1994) 

• Academic Decathlon Coaches Survey 
(April 1994) 

• Count of Medals Won 
(April 1994) 



1-5. Would Academic Decathlon continue if 
federal funds were not available? 



1-6. What was the Chapter 2 Formula cost 
per student (including the fixed cost of 
the program)? 



Program Coordinator Interview 
(April 1994) 

Academic Decathlon Coaches Survey 
(April 1994) 

Program Records (Spring 1994) 
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Educational 
Issue 
2 



LIBRARY RESOURCES AND INSTRUCTIONAL 
MATERIALS: Did the materials ordered by schools with Chapter 2 
Formula Funds enhance the educational experience of students? 



Date Needed: June 1994 



EVALUATION QUESTIONS 

2-1. How were funds allocated to schools? 
What types of items could be ordered? 



INFORMATION SOURCES 

• Grant Administrator Interview 
(January 1994) 



2-2. What materials were ordered by schools? 



Program Records (May 1994) 



2-3, How did staff rate the effectiveness of 
the materials purchased with Chapter 2 
Formula funds? 



Staff Survey (Spring 1994) 



2-4, In what ways did materials purchased 
with Chapter 2 Formula funds enhance 
the educational experience of students? 



Staff Survey (Spring 1994) 



2-5. What was the Chapter 2 Formula cost 
per student? 



Program Records (May 1994) 
Student Records Counts (June 
1994) 
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Educational 
Issue 
3 



MEGASKELLS: Does participation in the MegaSkills program by 
parents contribute to academic excellence in their school-age children? 
Should the MegaSkills program be continued in its present form? 



EVALUATION QUESTIONS 

3-1. How was the MegaSkills program 
implemented in 1993-94? 

3-2. How many parents were served by the 
MegaSkills Program? 

3-3. Did participation in the program by 

parents have a positive effect on student: 

• Attendance? 

• Achievement? 

• Discipline rates? 

• Retention? 

• Dropout status? 

3-4. Were student effects from last year's 
program sustained over time? 

3-5. How did participation in the program 
affect parental attitudes towards school? 

3-6. To what extent did parents feel the 
MegaSkills program facilitated 
parent-child dialogue? About which 
topics? 

3-7. How were MegaSkills "recipes" used at 
home? Which were most helpful? 

3-8. What was the Chapter 2 Formula cost 
per parent? Per student? How 
cost-effective was the program? 



Date Needed: June 1994 

INFORMATION SOURCES 



• Program Coordinator Interview 
(February 1993) 

• Program Records (Ongoing) 

• Parent Evaluation Form (Ongoing) 



• GENeric Evaluation SYStem 
(GENESYS) (Summer 1994) 

• Parent Evaluation Form (Ongoing) 



GENESYS (Summer 1994) 



Parent Survey (May 1994) 



Parent Survey (May 1994) 



Parent Survey (May 1994) 



• Program Records (May 1994) 

• GENESYS (Summer 1994) 
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Educational 
Issue 
4 



MIDDLE SCHOOL "TRANSITION IN" PROGRAM: 

Did the "Transition In" program facilitate the 
transfer of students to a new school setting? 



Date Needed: June 1994 



EVALUATION QUESTIONS 

4-1. How was the "Transition In" program 
implemented in 1993-94? 



INFORMATION SOURCES 

• Grant Administrator Interview 
(February 1994) 



4-2. Which students were served by the 
"Transition In H program? 



Program Records (April 1994) 



4-3. What services were provided at which 
schools? 



Staff Survey (April 1994) 



4-4. How did students and staff rate the 
effectiveness of the "Transition In" 
program? 



• Student Survey (April 1994) 

• Staff Survey (April 1994) 



4-5. What was the Chapter 2 Formula cost per 
student? 



• Program Records (June 1994) 
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Educational 
Issue 
5 



FULL-DAY PREKINDERGARTEN: Does supplemental instruction 
contribute to student achievement? Should the Full-Day 
Prekindergarten Component be continued in its present form? 



Date Needed: June 1994 
Evaluation Questions 5-7 through 5-8 will be addressed by the Chapter 1 evaluation. 



EVALUATION QUESTIONS 

5-1. How many students were served by the 
regular, bilingual, and ESL pre-K 
classes? By age, sex, ethnicity, and 
schools? By fiill-day and half-day 
classes? 



INFORMATION SOURCES 

• Attendance File (Ongoing) 

• Student Master File (STUD) (Ongoing) 



5-2. What was the attendance rate for pre-K 
students? Did this vary by type of class? 
By full day /half day? 



Attendance File (Ongoing) 



5-3. What were students* gains on the 
Peabody Picture Vocabulary Test- 
Revised (PPVT-R) and the TVIP (Test de 
Vocabulario en Imagenes Peabody) from 
fall (pretest) to spring (posttest)? How 
did gains compare to previous years? To 
the national norm? 



Peabody Picture Vocabulary Test-Revised 
(PPVT-R) File (September 1993, April 
1994) 

Test de Vocabulario en Imagenes Peabody 
(TVIP) File (September 1993, April 1994) 



5-4. How did the PPVT-R pre- to posttest 
gains compare: 

• In the fiill-day and half-day classes? 

• In the bilingual and English classes? 

• With low-income students? 

• Across different types of classes for 
students with the lowest pretest scores? 



PPVT-R File (September 1993, April 
1994) 

TVIP File (September 1993, April 1994) 



5-5. For 5 ^anish-speaking LEP students who 
took the Spanish TVIP and the English 
PPVT-R, how did pre- to posttest gains 
compare? 



PPVT-R File 

TVIP File 

LEP File (Ongoing) 
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Educational 

Issue 5 
Continued 



EVALUATION QUESTIONS 

5-6. Was there any difference between the 
achievement gains of full-day and 
half-day bilingual students? 

5-7. Were there differences in the achievement 
gains of pre-K students at different 
campuses? 



INFORMATION SOURCES 

• PPVT-R/TVIP File (Sept. /Oct. 1993, April 
1994) 

• LEP File (Ongoing) 



• PPVT-R/TVIP (Sept./Oct.l993, April 1994) 

• Student Master File (SMF) 



5-8. What were the strengths and the areas in • Coordinator Interview (May 1994) 

need of improvement in the imple- 
mentation of the pre-K component? 



Evaluation Question 5-9 will be addressed by the Chapter 2 evaluation. 



5-9. What was the Chapter 2 Formula cost per • Program Records (May 1994) 

student? How cost-effective was the 
program? 
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Educational 
Issue 
6 



PRIVATE SCHOOLS: How were Chapter 2 Formula funds used by 
private schools? 



Date Needed: June 1994 

EVALUATION QUESTIONS INFORMATION SOURCES 

6-1. How were private schools notified of • Grant Administrator Interview 

their eligibility for Chapter 2 Formula (January 1993) 
funds? How many were notified? How 
many elected to participate? 



6-2. What was the Chapter 2 Formula 

allocation per private school? How many 
students were served? By grade? 



6-3. How were Chapter 2 Formula funds 
utilized? How did private schools rate 
the effectiveness of library materials, 
computer software and hardware, 
instruction materials, and staff 
development funded by Chapter 2 
Formula? 



• Grant Administrator Records 
(January 1993) 

• Private School Survey (April 
1994) 

• Private School Survey (April 
1994) 



6-4. What educational experiences were 
provided by Chapter 2 Formula funds 
that would not otherwise be provided? 



• Private School Survey (April 
1994) 



6-5. What was the Chapter 2 Formula cost per • Private School Survey (April 1994) 

student? 
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Educational 
Issue 
7 



READING RECOVERY PROGRAM: Should the Reading Recovery 
Program in AISD be continued in its present form? 



Date Needed: June 1994 



Evaluation Questions 7-1 through 7-9 will be addressed by the Chapter 1 evaluation. 



EVALUATION QUESTIONS 



7-1. How many grade one students were 
served in the second year of 
implementation? What was the average 
length of stay? 

7-2. What percentage of children served by 
the Reading Recovery Program 
Districtwide were successfully exited 
from the program? 

7-3. How did successfully exited Reading 
Recovery students served by Chapter 1 
perform on a nationally normed 
standardized test? 

7-4. What was the rate of promotion for 
students who were exited from the 
Reading Recovery program compared 
with the District promotion rate? 

7-5. What responses to the program were 
made by teachers, administrators, 
Teacher Leaders, parents, and children? 



7-6. How cost-effective is the Reading 
Recovery program compared to other 
supplementary reading programs? 

7-7. What is the retention rate for trained 
teachers (from 1992-93 to 1993-94)? 
How does this compare to other urban 
districts? 



INFORMATION SOURCES 



• ROSS Forms 



• ROSS Forms 



• Iowa Tests of Basic Skills (TTBS) File 
(April) 

• Metropolitan Readiness Test (MRT) File 
(Fall 1993) 

• Student Master File 
(Promotion/Retention/Placed) 

• ROSS Forms 



• Survey of Reading Recovery Teachers and 
Classroom Teachers 

• Interviews with Teacher Leaders and 
Principals 

• Student Master File 

• General Ledger File 

• ITBS File (April) 

• Employee Master Record (EMR) File 

• AISD Teacher Leaders 

• U.S. Reading Recovery Headquarters at 
Ohio State University 
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Educational 
Issue 7 
Continued 



EVALUATION QUESTIONS 



INFORMATION SOURCES 



7-8. Are the effects of Reading Recovery 
sustained in the second grade for 
students who were exited from Reading 
Recovery in the first grade? 



• ITBS Reading Comprehension Texts (April 
1994) 

• Rank Order of Students, by Second Grade 
Teachers, According to Skills 



7-9. Has teacher training been sufficient for 
proper implementation of the program? 



• Teacher Surveys 

• Teacher Leader Interviews 



Evaluation Questions 7-10 through 7-12 will be addressed by the Chapter 2 evaluation. 



7-10. How was the training applied during 
sessions with Reading Recovery 
students? With regular classroom? 

7-11. Would this program continue if federal 
funds were not available? 

7-12. What was the Chapter 2 Formula cost / 
per participant? 



Classroom Observations (Spring 1994) 



• Program Coordinator Interview (February 
1994) 



Program Records (May 1994) 
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Educational 
Issue 
8 



READING RECOVERY TEACHER LEADER IN 
TRAINING AND CAMPUS LOCATIONS: Is the Reading 
Recovery Teacher Leader training effective? Should 
this training be continued in its present form? 



Date Needed: June 1994 



EVALUATION QUESTIONS 



8-1. How were Chapter 2 Formula funds 
used? What was the total cost of 
training? 



INFORMATION SOURCES 
• Program Records (May 1994) 



8-2. What type of training was received? 
Over what period of time did it take 
place? Who was trained? Where? By 
Whom? 



Staff Interview (Spring 1994) 



8-3. How effective was the training for 
Reading Recovery Teacher Leaders? 



Staff Interview (Spring 1994) 



8-4. What was the Chapter 2 Formula cost per 
participant? 



Program Records (May 1994) 
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Educational 
Issue 
9 



SPANISH ACADEMY: Does the provision of this training 
improve students' educational experience by enabling staff to deal 
more effectively with Spanish-speaking students and parents? Should 
the program be continued in its present form? 



Date Needed: June 1994 



EVALUATION QUESTIONS 



INFORMATION SOURCES 



9-1. How was the Spanish Academy 
implemented in 1993-94? 



Staff Interview (January 1994) 



9-2. How were classes advertised? How 
many courses were offered? How many 
AISD personnel attended a Spanish 
Academy course? How many have 
previously attended a course? What 
positions did they hold? Which level of 
instruction was most used by staff 
(beginner, intermediate, advanced)? 



Staff Interview (January 1994, February 
1994) 

Program Applications (January 1994, 

February 1994) 

Course Evaluation (Ongoing) 



9-3. Did participants view the course as 
effective? Did participation help staff 
communicate with Spanish-speaking 
students, parents, community members, 
and other staff? 



Course Evaluation (Ongoing) 



9-4. In what ways did participants find their 
Spanish stills most helpful? 



Course Evaluation (Ongoing) 



9-5. What was the Chapter 2 Formula cost 
per participant? 



Program Records (May 1994) 
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Educational 
Issue 
10 



STAFF DEVELOPMENT: How were Chapter 2 Formula funds used 
for staff development? 



Date Needed: June 1994 



EVALUATION QUESTIONS 



INFORMATION SOURCES 



10-1. What types of staff development were 
offered during 1993-94? To whom 
were thes? services offered? 



Grant Administrator Interview (April 1994) 



10-2. What materials were purchased with 
Chapter 2 Formula funds? 



Grant Administrator Interview (April 1994) 



10-3. What was the Chapter 2 Formula cost 
per staff member? 



Program Records (April 1994) 
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Educational 
Issue 
11 



STUDENT RETREAT" PROGRAM: How did the Student Retreat 
Program contribute to AISD's goal of drug-free schools? 



Date Needed: June 1994 



Evaluation Questions 11-1 through 11-7 will be addressed by the Drug-Free Schools evaluation. 



EVALUATION QUESTIONS 

11-1. What is the Student Retreat Program? 
How was it implemented? What 
services were provided? Who was 
served? 



INFORMATION SOURCES 

• Program Records (Ongoing) 

• Program Staff Interviews (Ongoing) 

• Attendance Files (Ongoing) 



11-2. What were the program's stated 
objectives? 



Program Records (Ongoing) 
Program Staff Interviews (Ongoing) 



11-3. What services were specifically directed 
toward decreasing and/or preventing 
alcohol and other drug use in AISD? 



• Program Records (Ongoing) 

• Program Staff Interviews (Ongoing) 

• Project Specialist Interviews (Ongoing) 



11-4. How successful was the program in 
meeting its stated objectives? 



11-5. How effective was the program 

perceived to be by students, teachers, 
staff, and Student Retreat Program staff 
with regard to decreasing and/or 
preventing alcohol and other drug use in 
AISD schools? 



Program Manager Interviews 
(Ongoing) 

GENESYS (Summer 1994) 

Staff Survey (Spring 1994) 
Substance Use Survey (Spring 1994) 
ORE Coordinated Survey (Spring 1994) 



11-6. How cost-effective was the Student 
Retreat Program? 



• Program Budget Records (Ongoing) 

• Grant Administrator Interview (Spring 
1994) 

• GENESYS (Spring 1994) 

• Substance Use Survey (Spring 1994) 



ERLC 
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Educational 
Issue 11 
Continued 



EVALUATION QUESTION 



INFORMATION SOURCES 



1 1-7. What is the long-term impact 
of the Student Retreat Program? 



• Archival DFS Program Participation 
Records (Ongoing) 

• GENESYS (Summer 1994) 



Evaluation Question 11-8 will be addressed by the Chapter 2 evaluation. 



11-8. How many times was the Student 
Retreat Program staff used for crisis 
intervention on campuses? On which 
campuses? 



Crisis Intervention Records (Ongoing) 



ERLC 
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Educational 
Issue 
12 



SUPPLEMENTARY MATERIALS: What training materials were 
purchased with Chapter 2 Formula funds? What type of training was 
offered to teachers? How was this training implemented within AISD? 



Date Needed: June 1994 



12-1. 



12-2. 



12-3. 



12-4. 



12-5. 



EVALUATION QUESTIONS 

What materials did Chapter 2 Formula 
provide to teachers for training? 



12-6. 



How did teachers rate the effectiveness 
of training and materials provided with 
Chapter 2 Formula funds? 

What type of training was offered to 
teachers? What specific techniques 
were taught? 

How was both Project Read and MTA 
training used with students: 

• One on one? 

• Small Groups? 

• Special Education? 

• In regular classrooms? 

To what extent were the materials and 
training utilized within AISD? Were 
MTA and Project Read implemented as 
designed: 

• By teachers trained in 1992-93? 

• By teachers trained in 1993-94? 

What type of materials and/or training 
would be provided if grant funds were 
not available? 



12-7. Which schools participated in the 
training? Which teachers? 

12-8. What was the Chapter 2 Formula cost 
per teacher? Per student? 



INFORMATION SOURCES 

• Grant Administrator Interview 
(January 1994) 

• Training Packets (Spring 1994) 

• Observation of Training Sessions (December 
1993, February, March, April 1994) 

• Teacher Survey (Spring 1994) 



• Observation of Training Sessions (December 
1993, February, March, April 1994) 



• Classroom Observations (Spring 1994) 

• Teacher Survey (Spring 1994) 



• Classroom Observations (Spring 1994) 

• Teacher Survey (Spring 1994) 



Grant Administrator Interview (Spring 1994) 



• Program Records (Spring 1994) 



• Program Records (Spring 1994) 

• Student Records Counts (June 1994) 
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Texas Education Agency (TEA) Report 1993-94 



Date Needed: June 1994 



INFORMATION QUESTIONS 

1. How many students were served, by 
grade, by Chapter 2 Formula, for each 
of the programs in the seven target 
assistance areas? What was the dollar 
amount budgeted for each of these 
programs? 

2. How many teachers, counselors and 
parents were served by Chapter 2 
Formula? What was the dollar amount 
budgeted for these services? 

3. What was the dollar amount budgeted 
for administration of the Chapter 2 
program including indirect costs? 

4. How many students were served, and 
what dollar amount was budgeted for 
private, nonprofit schools? 



INFORMATION SOURCES 

• Program Records (Ongoing) 

• Grant Administrator Records (Spring 1994) 



• Program Records (Ongoing) 

• Grant Administrator Records (Spring 1994) 



Grant Administrator Records (Spring 1994) 



Program Records (Ongoing) 
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DRUG-FREE SCHOOLS 



EVALUATION PLAN 



Contact Person: 
James Wiehe 



I 
I 



PROGRAM DESCRIPTION 

This is the seventh year ia which AISD will 
receive federal funds under the terms of the 
Drug-Free Schools and Communities (DFSC) Act 
of 1986 (P.L. 99-570) for the purpose of 
supplementing District efforts to eliminate the use 
of alcohol and other drugs on school campuses. In 
1993-94 AISD will receive $466,151 from the 
Drug-Free Schools and Communities grant, while 
approximately $118,806 is being rolled forward 
from the 1992-93 budget, for a total allocation of 
$584,957. Components included for funding for 
the 1993-94 school year include: 

Student Programs 

• Peer Assistance and Leadership (PAL) 
program; 

• Drug Abuse Resistance Education (DARE); 

• Student Retreat Program (formerly Student 
Alcohol and Drug Abuse Education and 
Prevention Program); and 

• Individual Campus-Based Programs; 

Staff Training and Curriculum Development 

• PreK-12 Curriculum Supplement; and 

• Student Assistance Program (SAP) Training; 

Support Staff and Services 

• Half-time wellness coordinator; 

• Quarter-time clerk to keep track of Prevention 
and Remediation In Drug Education (PRIDE) 
lat usage; 

• Full-time project facilitator; 

• Full-time budget control specialist; 

• Full-time evaluation associate; and 

• Supplemental Support Services. 

DFSC monies v/Ul also be allocated to private 
nonpublic schools within District boundaries, as 
required by law. Evaluation questions for the 
1993-94 school year will focus on cost 
effectiveness, the impact of student programs and 
staff training, and coordination of efforts across 
programs. 
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Educational I How were DFSC funds allocated? Were the programs which the funds 
Issue 1 were to support carried out? What overall impact did programs have on 
1 I students' alcohol and other drug use? 



Date Needed: 



EVALUATION QUESTIONS 

1-1. What programs, services, and materials 
were fiinded by the Drug-Free Schools 
and Communities (DFSC) grant? How 
did campuses receive funding? What 
level of program or training implemen- 
tation occurred in 1993-94 after 
receiving grant monies? Were the 
monies spent as originally allocated? 
What schools participated in which 
programs? 



June 1994 



INFORMATION SOURCES 

• Program Records (Ongoing) 

• Budget Records (Ongoing) 

• Project FacUitator Interviews (Ongoing) 

• Grant Administrator Interviews (Ongoing) 



1-2. What was the overall impact of the 
programs implemented during the 
1993-94 school year? To what extent 
was the overarching objective of 
decreasing and preventing alcohol and 
other drug use attained? 



• Project Facilitator Interview (Spring 1994) 

• Program Records (Ongoing) 

• Program Manager Interview (Spring 1994) 

• GENeric Evaluation SYStem (GENESYS) 
(Spring 1994) 

• Substance Use Survey (Spring 1994) 



1-3. Were there sufficient programs, services, 
and materials available to meet these 
objectives? How did each component 
contribute toward meeting these 
objectives? To what degree did the 
programs complement one another? 
Were they cost-effective? 



• Project Facilitator Interviews (Ongoing) 

• Program Records (Ongoing) 

• Budget Records (Ongoing) 

• Program Manager Interviews (Ongoing) 

• GENeric Evaluation SYStem (GENESYS) 
(Spring 1994) 

• Substance Use Survey (Spring 1994) 

• Advisory Council Representative Interview 
(Spring 1994) 



1-4. How were the program^ promoted? To 
what degree were students, teachers, 
counselors, and other program staff 
aware of the program components and 
training opportunities? 



• Project Facilitator Interview (Spring 1994) 

• Program Records (Ongoing) 

• Program Manager Interview (Spring 1994) 

• ORE Coordinated Survey (Spring 1994) 

• Substance Use Survey (Spring 1994) 
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Educational How did supplemental curricula, materials, and services purchased with 
issue Drug-Free Schools funds contribute to AISD's goal of drug-free 
2 schools? 



Date Needed: June 1994 



EVALUATION QUESTIONS INFORMATION SOURCES 

2-1. What supplemental curricula, materials, • Grant Administrator Interviews (Ongoing) 

and services were funded by the DFSC • Program Records (Ongoing) 
grant during 1993-94? 



Who was served by the supplemental 
curricula, materials, and services? 



• Project Facilitator Interviews (Ongoing) 

• Program Records (Ongoing) 

• Program Manager Interviews (Ongoing) 

• GENESYS (Summer 1994) 

• District Records (Spring 1994) 



2-3. How was the use of supplemental 
curricula, materials, and services 
monitored? 



• Project Facilitator Interviews (Ongoing) 

• Program Records (Ongoing) 

• Program Manager Interviews (Ongoing) 

• Campus-level Staff Interviews (Ongoing) 



2-4. How effective did teachers and other • Project Facilitator Interview (Sprin- 1994) 

staff believe the supplemental curricula, • ORE Coordinated Survey (Spring 1994) 

materials, and services were in 
preventing student alcohol and other 
drug use? 



2-5. How effective did students believe the • Program Student Surveys (Ongoing) 

supplemental curricula, materials, and • Substance Use Survey (Spring 1994) 

services were in preventing student 
alcohol and other drug use? 
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How effective were the 1993-94 Drug-Free Schools-ftmded program 
components and staff development activities in contributing to AISD's 
goal of drug-free schools? 



Date Needed: June 1994 



EVALUATION QUESTIONS 

3-1. How effective do administrators, 
teachers, other campus staff, and 
students believe programs funded 
through the DFSC grant are in helping 
the District address the problems of 
illegal drug and alcohol use among 
students? 



INFORMATION SOURCE 

• Project Facilitator Interviews (Ongoing) 

• ORE Coordinated Survey (Spring 1994) 

• Substance Use Survey (Spring 1994) 



3-2. Compared to other school issues, how • ORE Coordinated Survey (Spring 1994) 

does student alcohol and other drug use 
rank as a concern among AISD 
administrators, teachers, and other 
campus staff? 



3-3. Within DFSC-ftinded programs which 
have been implemented for three or 
more years, is there evidence that prior 
participants have experienced any 
long-term benefit from the programs? 



• Archival DFS Program Participation 
Records (Ongoing) 

• GENESYS (Summer 1994) 



3-4. What is the nature of alcohol and other 
drug use among AISD students 
compared with that of other students in 
the State and across the country? 



• National Surveys (Ongoing) 

• Statewide Surveys (Ongoing) 

• Substance Use Survey (Spring 1994) 
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How did Campus-Based Programs participate in the Drug-Free Schools 
program? 



Date Needed: June 1994 



EVALUATION QUESTIONS 

4-1. What are Campus-Based Programs! 
How did the campuses use the DFS 
funds allocated to them? Who was 
served through the individual campus 
DFS allocations? 



INFORMATION SOURCES 

• Principal Survey (Spring 1994) 

• Program Attendance Records (Ongoing) 



4-2. What were the prerequisites for funding 
any of the Campus-Based Programs! 



Grant Administrator Interview (Spring 1994) 
Project Facilitator Interviews (Ongoing) 



4-3. How effective do principals believe the 
DFS-funded activities/materials were in 
reducing or preventing student alcohol 
and other drug use? 



Principal Survey (Spring 1994) 



4-4. How cost-effective were the 

Campus-Based Programs! What was 
the impact of their supplemental 
activities and materials funded through 
the DFSC grant? 



Campus-Based Programs Participation 
Roster Sheets (Ongoing) 
Campus-Based Programs Budget Records 
(Ongoing) 

Grant Administrator Interview (Spring 1994) 
GENESYS (Summer 1994) 
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Educational 
Issue 
5 



How did the Peer Assistance and Leadership (PAL) program contribute 
to AISD's goal of drug-free schools? 



Date Needed: June 1994 
EVALUATION QUESTIONS INFORMATION SOURCES 



5-1. How was the PAL program implemented 
in 1993-94? What services were 
provided? Who was served? 



5-2. What were the program's stated 
objectives? 



Program Records (Ongoing) 
PAL Consultant Interviews (Ongoing) 
PAL Six-Weete r snorts (Ongoing) 
Attendance Files ;. : *»,Jing) 

Program Records (Ongoing) 

PAL Consultant Interviews (Ongoing) 



5-3. What services were specifically directed 
toward decreasing and/or preventing 
alcohol and other drug use in AISD? 



PAL Six-Weeks Reports (Ongoing) 
PAL Consultant Interviews (Ongoing) 
Project Facilitator Interviews (Ongoing) 



5-4. How successful was the program in 
meeting its stated objectives? 



PAL Consultant Interviews (Ongoing) 
GENESYS (Summer 1994) 
Campus Administrative Contact Interview 
(Spring 1994) 



5-5. How effective was the program 

perceived to be by PALs, PALees, and 
PAL instructors with regard to 
decreasing and/or preventing alcohol and 
other drug use in AISD schools? 



PAL Student Survey (Spring 1994) 
PALee Student Survey (Spring 1994) 
PAL/PALee Sponsor Survey (Spring 1994) 



5-6. How cost-effective was the PAL 
program? 



PAL Budget Records (Ongoing) 

Grant Administrator Interview (Spring 1994) 

GENESYS (Summer 1994) 

Substance Use Survey (Spring 1994) 

PAL Six-Weeks reports (Ongoing) 

Attendance Files (Ongoing) 



5^7. what is the long-term academic impact 
of the PAL program? 



• Archival PAL Program Records (Ongoing) 

• GENESYS (Summer 1994) 
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Educational 
Issue 
6 



How did the Drug Abuse Resistance Education (DARE) program 
contribute to AISD's goal of drug-free schools? 



Date Needed: June 1994 



EVALUATION QUESTIONS 



INFORMATION SOURCES 



6- 1 . How was the DARE program 

implemented? What services were 
provided? Who was served? 



Program Records (Ongoing) 
DARE Officer Interviews (Ongoing) 
Attendance Files (Ongoing) 



6-2. What were the program's stated 
objectives? 



• Program Records (Ongoing) 

• DARE Officer Interviews (Ongoing) 



6-3. What services were specifically directed 
toward decreasing and/or preventing 
alcohol and other drug use in AISD? 



Program Records (Ongoing) 
DARE Officer Interviews (Ongoing) 



6-4. How successful was the program in 
meeting its stated objectives? 



DARE Officer Interview (Spring 1994) 
DARE Officer Survey (Spring 1994) 
GENESYS (Summer 1994) 



6-5. How effective was the program 

perceived to be by students, teachers, 
staff, and DARE officers with regard to 
decreasing and/or preventing alcohol and 
other drug use in AISD schools? 



• DARE Officer Survey (Spring 1994) 

• Substance Use Survey (Spring 1994) 

• ORE Coordinated Survey (Spring 1994) 



6-6. How cost-effective was the DARE 
program? 



DARE Budget Records (Ongoing) 

Grant Administrator Interview (Spring 1994) 

GENESYS (Summer 1994) 

Substance Use Survey (Spring 1994) 



6-7. What is the long-term academic impact 
of the DARE program? 



Archival DARE Program Records (Ongoing) 
GENESYS (Summer 1994) 
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Educational 
Issue 
7 



How did the Student Retreat Program contribute to AISD's goal of 
drug-free schools? 



Date Needed: June 1994 
Evaluation Questions 7-1 through 7-7 will be addressed by the Drug-frze Schools evaluation. 

EVALUATION QUESTIONS INFORMATION SOURCES 



7-1. What is the Student Retreat Program! 
How was it implemented? What 
services were provided? Who was 
served? 



7-2, What were the program's stated 
objectives? 



7-3. What services were specifically direc ed 
toward decreasing and/or preventing 
alcohol and other drug use in AISD? 



7-4. How successful was the program in 
meeting its stated objectives? 



7-5. How effective was the program 

perceived to be by students, teachers, 
staff, and Student Retreat Program staff 
with regard to decreasing and/or ■ 
preventing alcohol and other drug use in 
AISD schools? 



• Program Records (Ongoing) 

• Student Retreat Program Program Staff 
Interviews (Ongoing) 

• Attendance Files (Ongoing) 



• Program Records (Ongoing) 

• Student Retreat Program Program Staff 
Interviews (Ongoing) 



• Program Records (Ongoing) 

• Student Retreat Program Program Staff 
Interviews (Ongoing) 

• Project Facilitator Interviews (Ongoing) 



• Student Retreat Program Program Manager 
Interview (Spring 1994) 

• GENESYS (Summer 1994) 



• Student Retreat Program Staff Survey 
(Spring 1994) 

• Substance Use Survey (Spring 1994) 

• ORE Coordinated Survey (Spring 1994) 



7-6. How cost-effective was the Student 
Retreat Program! 



Student Retreat Program Budget Records 
(Ongoing) 

Grant Administrator Interview (Spring 1994) 
GENESYS (Summer 1994) 
Substance Use Survey (Spring 1994) 
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Educational 

Issue 7 
Continued 



Date Needed: June 1994 



EVALUATION QUESTIONS 



INFORMATION SOURCES 



7-7. What is the long-term impact of the 
Student Retreat Program! 



• Archival DFS Program Participation 
Records (Ongoing) 

• GENESYS (Summer 1994) 



Evaluation Question 7-8 mil be addressed by the Chapter 2 evaluation. 



7-8. How many times was the crisis 

intervention team used? On which 
campuses? 



Crisis Intervention Records (Ongoing) 



77 

D-9 



93.07 



Educational 
Issue 
8 



How did the PreK-12 Curriculum Supplement contribute to AISD's goal 
of drug-free schools? 



Date Needed: June 1994 



EVALUATION QUESTIONS 

8-1. What was the PreK-12 Curriculum 

Supplement in 1993-94? What services 
and/or materials were provided? Who 
was served? How many teachers and/or 
students were served? 



INFORMATION SOURCES 

• Program Records (Ongoing) 

• PreK-12 Curriculum Supplement 
Administrator Interviews (Ongoing) 

• Attendance Files (Ongoing) 

• PRIDE Clerk Kit Usage Report (Spring 
1994) 

• PreK-12 Curriculum Supplement Service 
Records (Spring 1994) 



8-2. What was the intended purpose of the 
PreK-12 Curriculum Supplement? 



8-3. What supplemental services and/or 
materials were specifically directed 
toward decreasing and/or preventing 
alcohol and other drug use in AISD? 



Program Records (Ongoing) 
PreK-12 Curriculum Supplement 
Administrator Interviews (Ongoing) 

Program Records (Ongoing) 
PreK-12 Curriculum Supplement 
Administrator Interviews (Ongoing) 
Project Facilitator Interviews (Ongoing) 



8-4. How many Prevention and Remediation 
In Drug Education (PRIDE) kits were 
checked out? 



PRIDE Clerk Kit Usage Report (Spring 
1994) 



8-5. How successful was the PreK-12 

Curriculum Supplement in fulfilling its 
intended purpose(s)? 



• PreK-12 Curriculum Supplement 
Administrator Interview (Spring 1994) 



8-6* How effective was the supplemental 
curriculum perceived to be by students, 
teachers, staff, and the PreK-12 
Curriculum Supplement program manager 
with regard to decreasing and/or 
preventing alcohol and other drug use in 
AISD schools? 



Substance Use Survey (Spring 1994) 
ORE Coordinated Survey (Spring 1994) 
Pre K-12 Curriculum Supplement 
Administrator Interview (Spring 1994) 
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Educational 
Issue 
9 



How did Student Assistance Program (SAP) Training contribute to 
A USD's goal of drug-free schools? 



Date Needed: June 1994 
EVALUA TfON QUESTIONS INFORMATION SOURCES 



-1. How was SAP Training implemented in 
1993-94? What services were provided? 
Who was served? 



Program Records (Ongoing) 

SAP Training Program Manager Interviews 

(Ongoing) 

SAP Training Participation File (Ongoing) 
Trainee Follow-up Report of Students 
Served (Ongoing) 



9-2. What was the goal of implementing SAP 
Training? 



Program Records (Ongoing) 

SAP Training Program Manager Interviews 

(Ongoing) 



9-3. How was the SAP Training specifically 
directed toward decreasing and/or 
preventing alcohol and other drug use in 
AISD? 



Program Records (Ongoing) 

SAP Training Program Manager Interviews 

(Ongoing) 

Project Facilitator Interviews (Ongoing) 



9-4. How successful was SAP Training in 
accomplishing the goals set forth for it? 



SAP Training Program Manager Interview 
(Spring 1994) 



9-5. How effective was the SAP Training 
perceived to be by the trainees with 
regard to decreasing and/or preventing 
alcohol and other drug use in AISD 
schools? 

9-6. How cost-effective was SAP Training? 



SAP Training Workshop Evaluation Surveys 
(Ongoing) 



SAP Training Participation File (Ongoing) 
Trainee Follow-up Report of Students 
Served (Ongoing) 

SAP Training Budget Records (Ongoing) 
Grant Administrator Interview (Spring 1994) 
Substance Use Survey (Spring 1994) 
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Educational 
Issue 
10 



How did Supplemental Support Services equipment, supplies, materials, 
and consulting contribute to AISD's goal of drug-free schools? 



Date Needed: June 1994 
EVALUATION QUESTIONS INFORMATION SOURCES 



10-1. What Supplemental Support Services 
were funded through the 1993-94 DFS 
grant? What supplemental services were 
provided? How was the money spent? 
Who was served? 



Supplemental Support Services Records 
(Ongoing) 

Grant Administrator Interviews (Ongoing) 
AISD Police Program Manager Interviews 
(Ongoing) 



10-2. What was the purpose of the 

Supplemental Support Services! 



Supplemental Support Services Records 
(Ongoing) 

Grant Administrator Interview (Spring 1994) 



10-3. How successful were ^.Supplemental 
Support Services in accomplishing the 
goals set forth for them? 



Supplemental Support Services 
Participant/Contact Survey (Spring 1994) 



ERLC 



D-/2 



so 



93.07 



Educational 
Issue 
11 



How did the wellness coordinator contribute to AISD's goal of 
drug-free schools? 



Date Needed: June 1994 
EVALUATION QUESTIONS INFORMATION SOURCES 



11-1. What was the role of the Wellness 
Coordinator, as associated with 
Drug-Free Schools? What were the 
responsibilities associated with that 
position? 



• Wellness Coordinator Interview (April 1994) 

• Grant Adr inistrator Interview (April 1994) 



11-2. In what activities did the Wellness 

Coordinator engage during the 1993-94 
school year? What was accomplished? 



Wellness Coordinator Interview (April 1994) 
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How did the project facilitator contribute to AISD's goal of drug-free 
schools? 



Date Needed: 
EVALUATION QUESTIONS 



12- L What was the role of the Project 
Facilitator* as associated with 
Drug-Free Schools? What were the 
responsibilities associated with that 
position? 

12-2. In what activities did the Project 

Facilitator engage during the 1993 94 
school year? What was accomplished? 



June 1994 

INFORMATION SOURCES 

• Project Facilitator Interview (April 1994) 

• Grant Administrator Interview (April 1994) 

• Project Facilitator Interview (April 1994) 
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Educational 
Issue 
13 



How did the Private Nonpublic Sdiools participate in the 1993-94 DFS 
grant? 



Date Needed: June 1994 
EVALUATION QUESTIONS INFORMATION SOURCES 



13-1. What is the Private Nonpublic Schools 
component of the 1993-94 DFS grant? 
How was it implemented? What 
services were provided? Who was 
served? 



Program Records (Ongoing) 

Private Nonpublic Schools Recipients Survey 

(Spring 1994) 

Participation/Material Use Files (Ongoing) 



13-2. What were the stated objectives of the 
Private Nonpublic Schools'} 



• Program Records (Ongoing) 

• Private Nonpublic Schools Recipients Survey 
(Spring 1994) 



13-3. What services/materials were 

specifically directed toward decreasing 
and/or preventing alcohol and other drug 
use? 



Program Records (Ongoing) 

Private Nonpublic Schools Recipients Survey 

(Spring 1994) 



13-4. How successful were the Private 
Nonpublic Schools in meeting their 
stated objectives? 



Private Nonpublic Schools Recipients Survey 
(Spring 1994) 



13-5. How effective were the Private 

Nonpublic Schools efforts perceived to 
be by Private Nonpublic Schools 
students, teachers, and other staff with 
regard to decreasing and/or preventing 
alcohol and other irug use in their 
schools? 

13-6. How cost-effective were the Private 
Nonpublic Schools'} 



Private Nonpublic Schools Teacher, Student, 
and Other Staff Surveys (Spring 1994) 



Participation/Material Use Files (Ongoing) 
Private Nonpublic Schools Budget Records 
(Ongoing) 

Grant Administrator Interview (Spring 1994) 
Substance Use Survey for Private Nonpublic 
Schools (Spring 1994) 
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Information 
Needs 



TEA Report 1993-94 School Year 



Date Needed: June 1994 

The information needs are based upon questions asked in the 1992-93 TEA Report. Needs will be 
reassessed when requirements for the 1993-94 TEA Report are available, which in the past has been in 
April of the reporting year. 

Information Questions 1 through 7 are needed for both public and private nonprofit schools at the 
elementary and secondary levels. 



INFORMATION QUESTIONS 
1. What was the number of: 

• Counseling referrals for illicit drug 
and/or alcohol use? 

• Students referred for treatment related 
to illicit drug and/or alcohol use? 

• Disciplinary referrals related to 
possession or sale of illicit drugs 
and/or alcohol? 

• Juvenile arrests for offenses related to 
illicit drugs and/or alcohol? 

• Incidences of school-related gang 
violence (including verbal abuse, 
physical abuse, and destruction of 
property)? 

• Violent acts against students (including 
verbal abuse, physical abuse, and 
destruction of property)? 

• Violent acts against teachers/staff 
(including verbal abuse, physical 
abuse, and related destruction of 
property)? 



INFORMATION SOURCES 

• Counselor Report Form for DFS 
Accountability (Ongoing) 

• Discipline File (Spring 1994) 

• AISD Campus Police Report (Spring 1994) 
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Information 

Needs 
Continued 



Date Needed: June 1994 

Information Questions 1 through 3 are needed for both public and private nonprofit schools at the 
elementary and secondary levels. 



INFORMATION QUESTIONS 

1 . What was the number of: 

• Acts of vandalism against school 
property? 

• Expulsions related to illicit drug 
and/or alcohol use? 

• Students, by grade level, served by 
DFS programs during the 1993-94 
school year including K-3, 4-6, 7-y, 
and 10-12} 

2. How many of the students served by 
DFS programs during the 1993-94 
school year, were: 

American Indian or Alaskan Native? 
Asian or Pacific Islander? 
Black, not of Hispanic Origin? 
Hispanic? 

White, not of Hispanic Origin? 

3. Which of the following populations were 
targeted by DFS programs during the 
1993-94 school year: 

Students in general? 

Students at high risk for alcohol/drug use? 

Latchkey children? 

Student athletes? 

Out-of-school youth? 

Parents? 

Teachers and other school staff (not 

including counselors)? 
Counselors? 

Community groups /organizations? 
Law enforcement agencies? 



INFORMATION SOURCES 

• Program Records (Spring 1994) 

• Program. Attendance Files (Spring 1994) 

• GENESYS (Summer 1994) 

• Discipline File (Spring 1994) 

• AISD Campus Police Report (Spring 1994) 



Program Records (Spring 1994) 
Student Master File (Spring 1994) 



• Program Records (Spring 1994) 

• Project Facilitator Interview (Spring 1994) 
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Information 

Needs 
Continued 



Date Needed: June 1994 



Information Questions 4 through 6 are needed for both public and private nonprofit schools at the 
elementary and secondary levels. 



INFORMATION QUESTIONS INFORMATION SOURCES 



4. How much DFS money (in whole dollars) 
was spent in each of the following 
categories: 

Teacher/staff training? 
Student instruction? 

Curriculum development or acquisition? 
Student assistance programs (includes 

counseling, mentoring, and identification 

and referral)? 
Alternative education programs? 
Parent education/involvement? 
After-school recreation activities? 
Community service projects? 
Services for out-of-schooi youth? 
Special (one-time) events? 
Administration? 



• Program Records (Spring 1994) 

• Grant Administrator Interview/Report 
(Spring 1994) 



5. Did use of alcohol, tobacco, and other illicit • Substance Use Survey (Spring 1994) 

drugs decline from the 1992-93 school year? 



6. Did the following decrease from the 
1992-93 school year: 

The number of expulsions related to 
alcohol and other drugs? 

The number of disciplinary referrals 
related to alcohol and other drugs? 

The number of crimes against the 
community (related to alcohol and other 
drugs)? 



• Student Discipline Record (Spring 1994) 

• Travis County Juvenile Justice Report 
(Spring 1994) 
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Information 

Needs 
Continued 



Date Needed: June 1994 

Information Questions 7 and 8 are needed for both public and private nonprofit schools at the elementary 
and secondary levels. 



INFORMATION QUESTIONS 

Did the following decrease from the 
1992-93 school year (cont.): 

The number of counseling referrals to 
alcohol and other drug use? 

The amount of vandalism against 
school property? 

The number of incidences of school 
violence? 



8. Were there increases in the following during 
the 1993-94 school year: 

Parental involvement in school-related 
drug education and prevention 
activities? 

Teacher participation in staff 
development on alcohol and other 
drug use prevention? 

Community involvement in drug 
education and prevention activities 
from 1992-93? 



INFORMATION SOURCES 

• AISD Campus Police Report (Spring 1994) 

• Discipline Report (Spring 1994) 
o Counselor Report Form for DFS 

Accountability (Ongoing) 



• Program Records (Spring 1994) 

• Project Facilitator Interview (Spring 1994) 
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Information 

Needs 
Continued 



Date Needed: June 1994 

Information Questions 9 and 10 are needed for both public and private nonprofit schools at the 
elementary and secondary levels. 



INFORMATION QUESTIONS 

9. For the 1993-94 school year, was there 
evidence of improvement in the following 
academic areas: 

Student attendance? 
Number of tardies? 
Academic achievement? 
Dropout rate? 

Participation in cocurricular and 
extracurricular activities? 

10. At what grade levels (each grade, 
PreK-12, including EC) were the 
following DFS services clearly and 
consistently provided: 

Teacher/staff training? 
Student instruction? 

Curriculum development or acquisition? 
lent assistance programs (including 

counseling, mentoring, and identification and 

referral) ? 
Alternative education programs? 
Parent educationfinvolvement? 
After-school recreational activities? 
Community service projects? 
Service* for out-of-school youth (dropouts)? 
Special one-time events? 
Community involvement? 
Student surveys? 
Peer helping? 

Law enforcement support programs? 
Youth leadership training? 
Drug Abuse Resistance Education and related 
educational programs? 



INFORMATION SOURCES 

• Attendance Report (Spring 1994) 

• Attendance File (Spring 1994) 

• Absence File (Spring 1994) 

• Achievement Report (Spring 1994) 

• Dropout Report (Spring 1994) 

• To be determined (Spring 1994) 



Program Records (Spring 1994) 

Grant Administrator Interview (Spring 1994) 
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PROGRAM DESCRIPTION 



THE NATIONAL SCIENCE 
FOUNDATION GRANT TO 
THE SCIENCE 
ACADEMY: 1993-94 




Contact Person: 
Jeannine Turner 



The National Science Foundation (NSF) grant to 
the Science Academy of Austin was implemented 
in AISD beginning in 1990-91 under the name 
"The Austin Science and Mathematics 
Consortium: A Private Sector Partnership for 
Tomorrow's World/ This year is the fourth and 
final year of the grant. 

The project has two basic goals: 

1. To improve teaching skills (grades K-12) 
with technology tools that are available but 
underutilized, and 

2. To increase student learning and 
performance in science using holistic, 
interdisciplinary approaches with 
opportunities to apply concepts in real 
world settings. 

To address the identified goals, the 1993-94 NSF 
grant activities are divided into four components. 

• Curriculum Development: Each year of the 
grant has a specific focus on the curriculum 
development component. New curriculum- 
writing efforts are planned for 1993-94. 

• The Watershed Studies curricula will be 
developed for high school and piloted at 
the Science Academy during the 1993-94 
grant year. 

• The Science/Technology curricula 
developed during the 1991-92 grant year 
will continue to be piloted and modified as 
necessary at the Science Academy. 

• The Planet Earth curriculum developed 
during the 1990-91 grant year will 
continue to be piloted and modified as 
necessary at the Science Academy. 

• Get to the Point L a nonpoint source 
pollution curriculum for middle schools, 
developed during the 1990-91 grant year, 
piloted in fall 1991, and disseminated and 
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Program 
Description 
Continued 



utilized throughout the 10-county area served by 
the Lower Colorado River Authority since 
1992-93, will continue to be disseminated and 
modified as necessary. 

• Staff Development: During the summer, 
participating teachers attend training 
workshops. 

• Technology Institute : Teachers at LBJ 
High School and Winn Elementary School 
will be trained to integrate technology in 
their classrooms. Supplemental training 
and follow-up activities will take place 
during the school year. 

• River Watch Institute : Participating 
teachers (K-12) will learn river monitoring 
techniques and environmental action 
planning. 

• Student Participation: Students from the 
Science Academy will participate in the 
1993-94 grant activities. 

• Science Academy students will conduct 
outreach activities with the students of 
local elementary schools. 

• Science Academy students will assist in the 
training of students participating in the 
Colorado River Watch Network. 



• Private Sector Involvement/Linkages: 
Participants from the private sector will be 
extensively involved in all aspects of the 
grant. 

• Scientists, technologists, and engineers 
from the private sector will teach at the 
summer institutes and participate in follow- 
up activities. 

• A curriculum advisory team with 
representatives from local corporations will 
assist in the development of the new 
Watershed Studies curricula. 
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Decision 
Question 
1 



Should the Curriculum Development component be continued as is, 
modified, or discontinued? 



Date Needed: November 1994 



EVALUATION QUESTIONS 

1-1. How were the Planet Earth and Science/ 
Technology curriculum implemented at the 
Science Academy? How was the 
curriculum rated by Science Academy 
teachers? 

1-2. How was the nonpoint source pollution 
curriculum implemented in the 
participating schools? How was the 
curriculum rated by the participating 
teachers? 

1-3. What measures were developed to 

evaluate the following curricula developed 
by the grant? 

• Watershed Studies 

• Planet Earth 

• Science/Technology 

• Get to the Point! 

1-4. Did the curricula developed by the grant 
increase student interest, motivation, 
learning, and performance in science 
and mathematics concepts? 



INFORMATION SOURCES 

• Program Records (Ongoing) 

• Administrator Interview (Fall 1994) 

• Teacher Survey (Fall 1994) 



• Program Records (Ongoing) 

• Administrator Interview (Fall 1994) 

• Teacher Survey (Fall 1994) 



• Program Records (Ongoing) 

• Administrator Interview (Fall 1994) 



Teacher Survey (Fall 1994) 
Student Survey (Fall 1994) 
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Decision 
Question 
1 



Should the Staff Development component be continued as is, modified, or 
discontinued? 



Date needed: November 1994 



EVALUATION QUESTIONS 

2-1. How many teachers attended the 

Technology Institute? What were their 
characteristics (sex, ethnicity, grade 
taught, level of computer experience)? 

2-2. How did teachers rate the training they 
received at the Technology Institute? 
How did attending the Technology 
Institute affect teachers* level of computer 
use? How did the teachers implement 
their training in their classrooms? Were 
there differences between Becker and 
other project sites (if any) in their 
implementation and usage of the 
technology training? 

2-3. How many teachers attended the River 
Watch Training Institute? What were 
their characteristics (sex, ethnicity, grade 
taught)? 

2-4. How did participating teachers rate the 
training they received at the River Watch 
Institute? How actively did the teachers 
continue their river monitoring activities 
throughout the year? 



INFORMATION SOURCES 
• Program Records (Ongoing) 



• Teacher Survey (Summer 1994) 

• Program Staff Observations (Spring 1944) 



Program Records (Ongoing) 



• Program Records (Ongoing) 

• Teacher Survey (Summer 1994) 

• Monitoring Data Records (Ongoing) 
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Decision 
Question 
1 



Should the Student Participation component be continued as is, modified, 
or discontinued? 



Date Needed: November 1994 



EVALUATION QUESTIONS 

3-1. How many students participated in the 
River Watch water quality monitoring 
activities? What were their characteristics 
(sex, age, grade level)? 

3-2. How did their participation in the River 
Watch Network impact their environ- 
mental awareness? How did their 
participation enhance their science and 
mathematics motivation and achievement? 
What other skills did they develop? How 
did their participation r^ect their interest 
in science? 

3-3. How many Sci =,e Academy students par- 
ticipated ia the outreach activities with 
elementary schools? What were their 
characteristics (sex, age, grade level)? 
What kinds of activities did they conduct? 

3-4. How many Science Academy students 
participated in the River Watch student-to- 
student training activities? In what 
training activities? In what other River 
Watcn activities did Science Academy 
students participate? 



INFORMATION SOURCES 

• Program Records (Ongoing) 

• Student Master File (Fall 1994) 



• Teacher Survey (Fall 1994) 

• Student Survey (Fall 1994) 

• ITBS/NAPT File (Fall 1994) 



Program Records (Ongoing) 



• Program Records (Ongoing) 

• Administrator Interview (Fall 1994) 



E ' 5 93 



93.07 



Decision 
Question 
1 



Should the Private Sector Involvement/Linkages component be continued 
as is, modified, or discontinued? 



Date needed: November 1994 



EVALUATION QUESTIONS 

4-1. What linkages were developed among 
teachers, students, university faculty, 
and private sector leaders? What was 
the result (grants, curricula, donations, 
etc.) of these linkages? 

4-2. How many private sector participants were 
there? From which companies and/or 
organizations? In which aspect(s) of the 
grant was each involved? 



INFORMATION SOURCES 

• Program Records (Ongoing) 

• Administrator Interview (Fall 1994) 



• Program Records (Ongoing) 

• Administrator Interview (Fall 1994) 
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TITLE II MATHEMATICS 
AND SCIENCE 
TEACHER TRAINING 



EVALUATION PLAN 



Contact Person: 
Jeannine Turner 



PROGRAM DESCRIPTION 

Elementary and Secondary Education Act (ESEA) 
Title II is a 12-month project for the improvement 
of mathematics and science teaching in grades pre- 
K through 12. Title Il-funded projects are 
designed to serve all elementary and secondary 
mathematics and science teachers, and their 
students, in the Austin Independent School District. 
From July 1, 1993, to June 30, 1994, Title II 
funds will provide: 

• Staff development workshops to provide 
districtwide training for teachers to: 

• Become aware of new instructional and 
assessment techniques, 

• Use hands-on materials appropriately, and 

• Have ongoing opportunities for self- 
improvement. 

• Consultants to modify the existing 
mathematics and science curricula by 
integrating higher-order analytical and 
problem-solving skills. 

• Funds to supplement District efforts to 
improve mathematics and science 
instruction for all students in grades 6-9, 
with special emphasis on historically 
underrep resented groups. 

• Funds for teachers to attend professional 
conventions and conferences to provide for 
teacher involvement within their profession 
and expose teachers to current trends, 

• A mathematics specialist to conduct staff 
development workshops for teaching strategies, 
instructional techniques, and assessment 
techniques with special emphasis for teachers of 
grades 6-9. 

• Materials to accompany training in new 
methods of instruction, 
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Program 
Description 
Continued 



• Tuition/stipends for teachers to attend staff 
development workshops, and 

• An evaluation contractor to document and 
report on the effectiveness of project activities 
and to provide evaluation information required 
by the State and the District. 



For the 1993-94 school year, AISD has received 
$342,157 in Tide II funds (which includes 
$135,390 rolled forward from 1992-93). The 
allocations are for a mathematics specialist 
($82,261), elementary science ($65,849), middle 
school science ($21,487), secondary science 
($41,729), elementary mathematics ($46,778), 
middle school mathematics ($42,483), secondary 
mathematics ($26,455), nonpublic schools 
($2,945), evaluation ($6,086), management 
($1,750), and indirect costs ($4,337). 

T ->als for the use of the 1993-94 Tide II funds 



1. To supplement District efforts to improve 
mathematics and science instruction for all 
students in grades 6-9, with special emphasis 
on the historically underrepresented groups. 

2. To proviue ongoing districtwide training for 
teachers to become aware of new instructional 
and assessment techniques, to use hands-on 
materials appropriately, and to have ongoing 
opportunities for self-improvement. 

3. To modify the existing mathematics and 
science curricula by integrating higher-order 
analytical and problem-solving skills. 

Coordination and guidance for the project are 
provided by the AISD instructional coordinators for 
madiematics and science. 
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K ~ I Should District efforts to improve mathematics and science instruction 
issue I in grades 6-9 through the use of Tide II funds and in accordance with 
1 I Improvement Area II and Improvement Area III of the District's 
1993-94 improvement plan, continue as is, be modified, or 
discontinued? 



Date Needed: June 1994 



EVALUATION QUESTIONS 

1-1. What was done to improve 

mathematics and science instruction 
in grades 6-9? 

1-2. How was special emphasis given to 
historically underrepresented 
groups? 

1-3. In what ways did the Mathematics 
Specialist help to improve 
mathematics instruction in grades 
6-9? 

1-4. How did students* achievement 
(grades 6-9) on the NAPT 
mathematics and science subscales 
compare to their previous year's 
scores? 

1-5. How did historically underrepre- 
sented students* achievement 
(grades 6-9) on the NAPT 
mathematics and science subscales 
compare to their previous year's 
scores? 



INFORMA HON SOURCES 

• Administrator Interviews (Spring 1994) 

• Project Records (Ongoing) 



• Administrator Interviews (Spring 1994) 

• Project Records (Ongoing) 

• Administrator Interviews (Spring 1994) 

• Project Records (Ongoing) 



• Norm-referenced Assessment Program for 
Texas (NAPT) File (May 1994) 



• NAPT File (May 1994) 
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Should Title Il-funded staff training, in accordance with Improvement 
Area II of the District's 1993-94 improvement plan, continue as is, be 
modified, or discontinued? 



Date Needed: June 1994 



EVALUATION QUESTIONS 

2-1. What new instructional techniques 
were demonstrated to mathematics 
and science teachers? 

2-2. What new assessment techniques 
were demonstrated to mathematics 
and science teachers? 

2-3. What new hands-on materials were 
demonstrated to mathematics and 
science teachers? 



INFORMATION SOURCES 

• Administrator Interviews (Spring 1994) 

• Project Records (Ongoing) 



• Administrator Interviews (Spring 1994) 

• Project Records (Ongoing) 



• Administrator Interviews (Spring 1994) 

• Project Records (Ongoing) 
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Should District efforts to modify mathematics and science curricula by 
integrating higher-order analytical and problem-solving skills continue as 
is, be modified, or discontinued? 

Date Needed: June 1994 



EVALUATION QUESTIONS 

3-1. How were mathematics and science 
curricula altered to incorporate 
higher-order analytical and 
problem-solving skills? 

3-2. What was the role of consultants in 
modifying mathematics and science 
curricula to incorporate higher- 
order analytical and probleu- 
solving skills? 

3-3. How were teachers introduced to 
the curricula changes? 

3-4. For which grades and in what 
subject areas were the curricula 
changed? 



INFORMATION SOURCES 

• Administrator Interviews (Spring 1994) 

• Project Records (Ongoing) 

• Administrator Interviews (Spring 1994) 

• Project Records (Ongoing) 

• Administrator Interviews (Spring 1994) 

• Administrator Interviews (Spring 1994) 
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Information 
Needs 



TEA Report 1993-94 



Date Needed: June 1994 



INFORMATION QUESTIONS 

1. What was the number of participants 
served with Title II funds? How 
many of the participants were: 

• Teachers 

• Administrators and Supervisors 

• Other Staff? 

2. What was the gender and ethnicity of 
program participants? 

3. What was the number of elementary, 
noddle school /junior high, and high 
school teachers who participated in 
Title II-funded projects? 

4. How many participants became 
certified subject area specialists 
because of Title II-funded training? 
What type of certification did they 
receive? 



INFORMATION SOURCES 

• Project Records (Ongoing) 

• Wori^hop Sign-in Sheets (Ongoing) 

• Employee Master File (May 1994) 



• Project Records (Ongoing) 

• Workshop Sign-In Sheets (Ongoing) 

• Employee Master File (May 1994) 

• Project Records (Ongoing) 

• Workshop Sign-In Sheets (Ongoing) 

• Employee Master File (May 1994) 



• Administrator Interviews (Spring 1994) 
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Information 

Needs 
Continued 



INFORMATION QUESTIONS 

5. How many teachers received 
"individual grants" to fund projects 
designed to improve their teaching 
ability and/or improve mathematics 
and science instruction materials. 
What was the total amount of 
money devoted to this purpose? 

6. How many teachers participated in 
projects to meet special needs of 
underrepreaented groups? 

7. How many racial/ethnic minority 
teachers were recruited through the 
activities supported by Title II 
programs? 

8. How many students were directly 
affected as a result of the Title II- 
fimded programs? 

9. How many public elementary, 
middle, and high school campuses 
participated in Title II programs? 

10. How many campuses received 
computer equipment or 
telecommunications equipment 
purchased with Title II ftmds? 



INFORMATION SOURCES 
• Project Records (Ongoing) 



• Administrator Interviews (Spring 1994) 

• Project Records (Ongoing) 

• Administrator Interviews (Spring 1994) 



• Employee Master File (May 1994) 

• Student Master File (May 1994) 

• Project Records (Ongoing) 



• Project Records (Ongoing) 
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Information 

Needs 
Continued 



INFORMATION QUESTIONS INFORMATION SOURCES 



1 1 . What was the primary purpose of Title II- 
funded projects? What was the total 
amount of money devoted to that one 
purpose? 

12. Were any Title H-funded projects 
specifically designed to improve access to 
and participation in mathematics and 
science programs for any of the following 
underrepresented groups? 

• Female students? 

• Racial/ethnic minority students? 

• Students with limited English 
proficiency? 

• Economically disadvantaged students? 

• Handicapped students? 

• Migrant students? 

• Gifted and talented students from the 
above groups? 

15. What public and/or private sector groups 
did Title II-funded projects collaborate? 

14. What subject area (mathematics, science, 
or both) was primarily offered with Title 
II projects? 

15. What was the duration of most Title II 
projects? 



• Project Records (Ongoing) 



• Administrator Inter lews (Spring 1994) 

• Project Records (O igoing) 



• Administrator Interviews (Spring 1994) 

• Project Records (Ongoing) 

• Administrator Interviews (Spring 1994) 

• Project Records (Ongoing) 



• Administrator Interviews (Spring 1994) 

• Project Records (Ongoing) 
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TITLE W 
BILINGUAL EDUCATION 
TRANSITION PROGRAM 



EVALUATION PLAN 



Contact Person: 
Rosa Maria Gonzalez 



PROGRAM DESCRIPTION 

In 1990-91, AISD received a three-year Title VII 
grant to serve a population of limited English- 
proficient high school students called 
"newcomers. M In addition to being limited 
English proficient, these students have recently 
arrived in the United States. For the purpose of 
the program, a newcomer is defined as a student 
who has been in the U.S. for one year or less. 
The students typically have limited or interrupted 
schooling in their home countries and a wide 
range of literacy skills. Increasing numbers of 
immigrants continue enrolling at three high 
school campuses, Austin, Lanier and Reagan. 

The Title VII Program was funded for its second 
year during the summer of 1991, and for its third 
year in the summer of 1992. Some Title VII 
programs were given the opportunity to submit 
proposals requesting funding for a fourth year of 
implementation. In the fall 1992 semester, AISD 
submitted its proposal, and the request for a 
fourth year of operation was granted by U.S. 
Department of Education the latter part of the 
spring 1993 semester. The Title VII Program 
will continue with its established format as in 
previous years, with the added emphasis that 
includes following up former newcomers. 

The goal of the Title VII Program is to improve 
the English proficiency and the achievement skills 
of the target students. The program is designed 
to provide a sheltered environment for its 
participants. Class size is kept relatively small, 
and the students receive three hours of intensive 
English instruction daily which includes listening, 
speaking, reading, writing, grammar, and 
vocabulary development. In addition, students 
may enroll in a variety of subject areas to , 
complete their schedules. A teacher and teacher 
assistant are assigned to each of the three 
campuses. The intent of the program is for 
students to attend regular English as a Second 
Language (ESL) classes at the end of one year. 
Approximately 100 students will be served by the 
program during its fourth year of operation. 
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Should the Title VII Newcomers Program be continued as it is, 
modified, or discontinued? 



EVALUATION QUESTIONS 



INFORMATION SOURCES 



1-1. What were the characteristics of 
the Tide VII newcomers in terms 
of: 

• Number of students served? 

• Length of time in the U.S.? 

• Countries of origin? 

• Urban or rural setting? 

• Amount of schooling in home 
country? 

• Percent of students overage for grade? 

How did the Title VII newcomers 
served in 1993-94 compare to the 
students served in previous 
years? 

1-2. What selection criteria were used 
to identify program participants? 
Was the student background 
information assessed and 
disseminated to program teachers 
and content area teachers? 
How did the selection criteria 
compare to previous years' 
criteria? 



• Program Records (Ongoing) 

• Student Intake Forms 
(Fall 1993, Spring 1994) 



• Program Coordinator Interview 
(May 1994) 

• Teacher and Teacher Assistant Interviews 
(May 1994) 



1-3. To what extent were the 

following program objectives 
accomplished? 

• Computer-aided instruction utilized? 

• Cooperative learning activities 
utilized? 

• Activities for orientation to life in the 
U.S.? 

• In-service training for the content area 
teachers? 

• Specific activities to increase parental 
involvement? 



Program Coordinator Intertvtew 
(May 1994) 

Teacher and Teacher Assistant Interviews 
(May 1994) 

Program Records (Ongoing) 
Staff Surveys (April 1994) 
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Key 
Issue 1 
Continued 



EVALUATION QUESTIONS 

1-4. How much did the Title VII 

newcomers improve their acquisition of 
English as a result of the program? 

1-5. Did the Title VII newcomers 

improve their academic achievement as a 
result of the program? 

• How did their TAAS and NAPT 
scores compare to those of a 
comparison group? 

• How did they compare op. other 
measures of school success such as 
attendance, GPA, credits earned, and 
dropout rate? 

1-6. What did the Title VII newcomers think 
of the program? Did they feel welcome 
in their schools? Did the program foster 
confidence about staying in school? 

1-7. What additional services were 

provided to former newcomers exiting 
the program? 

• What is the academic and graduation 
status of former students? 

• To what extent have at-risk programs 
in the schools been utilized to keep 
former newcomers in school? 

• To what extent are former newcomers 
participating in extracurricular 
activities? 



INFORMATION SOURCES 

• English Language Assessment Battery (LAB) 
(Fall 1993, Spring 1994) 

• Student Survey (April 1994) 

• ORE GENeric Evaluation SYStem 
(GENESYS) (Summer 1994) 



Student Survey (January and April 1994) 



Program Records (Ongoing) 
GENESYS (Summer 1994) 
Former Student Survey 
(January and February 1994) 
Graduating Student Exit Interviews 
(May 1994) 

Program Coordinator Interview 
(May 1994) 

Teach nd Teacher Assistant Interviews 
(May 1994) 
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Key 
Issue 1 
Continued 



EVALUATION QUESTIONS 

1-8. What concerns and strengths about the 
implementation of the program were 
identified by: 

• Program administrators? 

• Program teachers? 

• Program teacher assistants? 

• Program LPAC chairpersons? 

1-9. How many parents attended the parent 
meetings sponsored by the program? 
What was their opinion of the meetings? 

1-10. What was the impact of the teacher 
assistant in the Title VII newcomers 
classroom? 

• Was their role clearly defined? 

• How were teacher assistants utilized? 

• Did the teacher and teacher assistant 
at each campus exhibit a H team 
approach" to instruction and other 
matters concerning newcomers? 

1-11. To what extent was the teacher assistant 
allowed to provide follow-up services to 
former newcomers? 

• Was the role of the teacher assistant 
clarUed to facilitate the follow-up of 
former students? 

• Were the additional tasks for the 
teacher disseminated among the 
appropriate school personnel? 

• What linkages was the teacher 
assistant able to establish between 
former newcomers and the at-risk 
programs? 



INFORMATION SOURCES 

• Teacher and Teacher Assistant Interviews 
(May 1994) 

• Staff Survey (April 1994) 

• Program Coordinator Interview 
(May 1994) 



• Attendance Records (Ongoing) 

• Evaluation Forms (Ongoing) 



• Staff Survey (April 1994) 

• Program Records (Ongoing) 

• Student Survey (January and April 1994) 

• Teacher and Teacher Assistant Interviews 
(May 1994) 



• Program Coordinator Interview (May 1994) 

• Teacher and Teacher Assistant Interviews 
(May 1994) 

• Program Records (Ongoing) 

• Staff Survey (April 1994) 

• Minutes from Staff Meetings 
(Fall 1993, Spring 1994) 
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Key 
Issue 1 
Continued 



EVALUATION QUESTIONS 

1-12. How many teachers and teacher assistants 
attended the training workshops sponsored 
by the program? What was their opinion 
of the workshops? 

1-13. To what extent did the teachers and 
teacher assistants use program funds 
available to attend college? What courses 
did they take? 

1-14. To what extent did the program teachers 
model: 

• Commitment to students? 

• Cultural sensitivity? 

• Ability to motivate others? 

• Ability to work well with the teacher 
assistant? 

• Innovative use of materials and 
equipment? 

• Ability to find utilize school and 
community resources? 

1-15. What was the 1993-94 budget for the Title 
VII Newcomers Program? 

• What did the fiinds provide? 

• What was the cost per student and per 
student contact hour? 

1-16. What was the cost of local support 
for the program? 



INFORMATION SOURCES 
• Evaluation Forms 



• Attendance Records 

• Evaluation Forms 



Program Records (Ongoing) 

Staff Survey (April 1994) 

Teacher and Teacher Assistant Interviews 

(May 1994) 

Program Coordinator Interview 
(May 1994) 



Program Budget (Fall 1993) 
Program Records (Ongoing) 



Program Records (Ongoing) 
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Key 
Issue 1 
Continued 



EVALUATION QUESTIONS 



INFORMATION SOURCES 



1-17. How effective was the in-school group 
counseling provided for program students 
in addressing issues of cultural adjustment? 



1-18. Was there any uniformity in the program's 
management and curriculum on the three 
campuses? 

• Was there good communication 
among the campuses to share ideas 
and solve problems? 

• Is there a system for documenting time 
spent on different classroom activities? 



• Teacher and Teacher Assistant Interviews 
(May 1994) 

• Program Coordinator Interview 
(May 1994) 

• Student Survey (January and April 1994) 

• Student Evaluations (Ongoing) 

• Teacher and Teacher Assistant Interviews 
(May 1994) 

• Program Coordinator Interview 
(May 1994) 

• Program Records (Ongoing) 



1-19. How was the testing of students and 
essential paperwork managed? 

• Was all LAB pre- and posttesting 
completed in a consistent and timely 
manner? 

• What alternative testing methods were 
utilized for the assessment of language 
acquisition? 

• Was all paperwork related to 
participants completed and submitted 
for computer entry in a consistent and 
timely manner? 

• Was there an efficient system for 
receiving and interpreting credits fron 
other countries for program students? 



• Teacher and Teacher Assistant Interviews 
(May 1994) 

• Program Coordinator Interview 
(May 1994) 

• Program Records (Ongoing) 
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78702 RAYS 
DROPOUT RECOVERY 
PROGRAM 



EVALUATION PLAN 



Contact Person: 
David Wilkinson 



PROGRAM DESCRIPTION 

On October 11, 1993, the Board of Trustees approved 
a contract with Middle Earth Unlimited, Inc. to serve 
eligible AISD recovered dropouts during the 1993-94 
school year. Middle Earth, along with AISD, the City 
of Austin, Communities in Schools, Austin Commu- 
nity College, Austin/Travis County Youth Services, 
Inc., Southwest Correctional Arts Network (SCAN), 
Texas Youth Commission (TYC), and the Central East 
Austin Community Organization (CEACO), is a 
partner in the Reaching All Youth Services (RAYS) 
Collaboration The RAYS Collaboration targets 
children and youth aged 0-16 and up to 21 for General 
Educational Development (GED) services. The 
geographic target area is east Austin with priority 
given to the Alternative Learning Center (ALC) stu- 
dents and their families, as well as children, youth, 
and families residing in zip code geographical area 
78702 and peripheral communities, and youth and 
families from the greater Austin community as 
appropriate. Reportedly, 80% of the residents in the 
78702 zip code area who are older than 25 years of 
age have neither a high school diploma nor a GED. 

RAYS has been housed in two portable classrooms on 
the ALC campus. The RAYS Collaboration is seeking 
to expand its services by offering a competency-based 
curriculum leading to a high school diploma during 
the 1993-94 school year. The program will be 
scheduled as an evening program on the ALC campus, 
operating in an additional two portables, and will 
serve up to 50 students. The purpose of the program 
is to provide an alternative for dropout recovery with 
the 78702 zip code geographical area as the primary 
target area. 

Eligibility for service is based on age, residence, and 
dropout status. Middle Earth will serve "only those 
students who... are considered dropouts under our 
District's dropout identification criteria." The contract 
specifies that Middle Earth will "provide appropriate 
instruction [to students] to assure the mastery of 
essential elements... and/or provide GED instruction, 
including pretesting and posttesting at a GED 
accredited testing center." Instruction is to be 
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Program 
Description 
Continued 



provided by certified staff who have at least a 
baccalaureate degree from an accredited college or 
university. Middle Earth is also to provide Texas 
Assessment of Academic Skills (TAAS) tutorial 
support. Program students are to participate in state- 
mandated testing (as required in TEC 21.551) and 
activities in AISD's Area 2. Middle Earth will use 
State-adopted textbooks supplied by AISD. With this 
program, the District hopes to reduce the AISD 
dropout rate and to provide another option for serving 
at-risk students in a very high need area. Through 
RAYS, the District could re-enroll these students in a 
diploma or GED preparation program. The District 
will award a diploma from the Evening High School 
to students upon successful completion of high school 
graduation requirements and allow those students to 
participate in graduation ceremonies at the Evening 
High School. The cost of the diplomas-estimated at 
$1 . 10 per diploma-is to be borne by Middle Earth. 

The District agreed to pay to Middle Earth each 
semester that portion of the State share of the 
Foundation School Program that is netted by the 
average daily attendance (ADA) of AISD students who 
attend the ALC under this contract minus the AISD 
processing fee, which is equal to the indirect cost rate 
established by the Texas Education Agency (TEA) for 
those funds-2.303% for 1993-94. The Net State Share 
is defined as the total State payment of ADA-based 
funds to AISD less die local fund assignment of AISD 
and all other deductions from state funds received by 
the District, so that the total amount paid to Middle 
Earth will not exceed the net amount of state funds 
received by the District reduced by the processing fee. 
The actual amount to be received by AISD, after local 
fund assignment and other deductions, is approxi- 
mately $900 per student in ADA. Therefore, the cost 
of the program is estimated as $45,000 (projected-state 
funds for 50 students generated by the RAYS 
Collaboration's ADA). 

Middle Earth is to receive a payment of 33% of the 
estimated funds due for the year at the end of the first 
and second semesters. This estimate will be based on 
the ADA generated by the end of each semester. A 
final payment of the balance due will be made at the 
end of the Middle Earth summer program. 
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Decision 
Question 
1 



Should the District continue to contract with Middle Earth Unlimited, Inc. 
and/or other community-based organizations for alternative education 
programs' for District dropouts? 
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Date Needed: August 

EVALUATION QUESTIONS 

1-1. How were students identified as 
eligible for service? 

1-2. How many students were 

contacted about participating in the 
program? 

1-3. How many students were served 
during each semester of 1993-94? 

1-4. What was the attendance rate each 
semester for program students? 

1-5. How many program students were 
awarded an AISD diploma from the 
Evening High School? 

1-6. How many students completed the 
requirements for a GED? 

1-7. What grades did participating 
students receive? 

1-8. On the average, how many course 
credits did participating students 
earn? 

1-9. How did program students perform 
on the Texas Assessment of 
Academic Skills (TAAS)? On the 
Norm-referenced Assessment 
Program for Texas (NAPT)? 

1-10. What did the program cost overall 
and per dropout recovered? 



1994 



INFORMATION SOURCES 

• Administrator Interview (February 1994) 

• Program Records (February 1994) 

• Program Records (February 1994) 



Program Records (May, August 1994) 



Program Records (May, August 1994) 



• District Records (May, August 1994) 

• Graduation File (August 1994) 



Program Records (May, August 1994) 



Program Records (May, August 1994) 



Program Records (May, August 1994) 



• TAAS File (May 1994) 

• NAPT File (May 1994) 



Program Budget (June, August 1994) 
• Program Records (May, August 1994) 
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AMERICAN INSTITUTE 
FOR LEARNING 
(AIL) DROPOUT 

RECOVERY PROGRAM 



EVALUATION PLAN 



Contact Person: 
David Wilkinson 



PROGRAM DESCRIPTION 

On August 9, 1993, the Board of Trustees 
approved a contract with the American Institute 
for Learning (AIL) to serve eligible AISD 
recovered dropouts during the 1993-94 school 
year. Eligibility for service is based on age, 
residence, and dropout status. The contract 
specifies that AIL will "provide appropriate 
instruction [to students] to assure the mastery of 
essential elements... and/or [assure that students] 
complete the requirement for a GED...." 
Instruction is to be provided by certified staff 
who have at least a baccalaureate degree from an 
accredited college or university. AIL is also to 
provide Texas Assessment of Academic Skills 
(TAAS) tutorial support. Program students are to 
participate in state-mandated testing (as required 
in TEC 21.551) and activities in AlSD's Area 4. 
AIL has three four-hour blocks of instruction- 
8:00 a.m.-12:00 noon, 1:00 p.m.-5:00 p.m., and 
5:00 p.m.-9:00 p.m. AIL will use State-adopted 
textbooks supplied by AISD. 

With this program, the District hopes to reduce 
the AISD dropout rate and to provide another 
option for serving at-risk students at the high 
school level. AIL is projected to serve an 
estimated 200+ AISD dropouts. Through AIL, 
the District could re-enroll these students in a 
diploma or GED preparation program. The 
District will award a diploma from the Evening 
High School to students upon successful comple- 
tion of high school graduation requirements and 
allow those students to participate in graduation 
ceremonies at the Evening High School. The cost 
of the diplomas-estimated at $1.10 per diploma- 
is to be borne by AIL. 

The District agreed to pay to AIL each semester 
that portion of the State share of the Foundation 
School Program that is netted by the average 
daily attendance (ADA) of AISD students who 



LIS 



1-1 



112 



93.07 



Program 
Description 
Continued 



attend AIL minus the AISD processing fee, which 
is equal to the indirect cost rate established by the 
Texas Education Agency (TEA) for those funds- 
2.303% for 1993-94. The Net State Share is 
defined as the total State payment of ADA-based 
funds to AISD less the local fund assignment of 
AISD and all other deductions from State funds 
received by the District, so that the total amount 
paid to AIL will not exceed the net amount of 
State funds received by the District reduced by 
the processing fee. The actual amount to be 
received by AISD, after local fund assignment 
and other deductions, is approximately $900 per 
student in ADA. Therefore, the cost of the 
program is estimated as $180,000 (projected state 
funds for 200 students generated by AIL's ADA). 

In addition to this cost, AISD agreed to provide 
transportation to eligible students to the 
alternative education program and to the site 
where students will participate in state-required 
testing- The cost for providing transportation for 
200 students utilizing Capital Metro bus passes is 
estimated at $6,000 ($5.00 per student per month 
for six months). AISD will also provide text- 
books and teacher editions according to State 
code. 

AIL is to receive a payment of 33% of the 
estimated funds due for the year at the end of the 
first and second semesters. This estimate will be 
based on the ADA generated by the end. of each 
semester. A final payment of the balance due 
will be made at the end of the AIL summer 
program. 
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Decision 
Question 
1 



Should the District continue to contract with the American Institute for 
Learning and/or other community-based organizations for alternative 
education programs for District dropouts? 

Date Needed: August 1994 

INFORMATION SOURCES 



EVALUATION QUESTIONS 

1-1. How were students identified as 
eligible for service? 

1-2. How many students were contacted 
about participating in the program? 

1-3. How many students were served 
during each semester of 1993-94? 

1-4. What was the attendance rate 
each semester for program 
students? 

1-5. How many program students were 
awarded an AISD diploma from the 
Evening High School? 

1-6. How many students completed the 
requirements for a GED? 

1-7. What grades did participating 
students receive? 

1-8. On the average, how many course 
credits did participating students 
earn? 

1-9. How did program students perform 
on the Texas Assessment of 
Academic Skills? On the 
Norm-referenced Assessment Program 
for Texas (NAPT)? 

1-10. What did the program cost overall 
and per dropout recovered? 



Administrator Interview (February 1994) 
Program Records (February 1994) 

Program Records (February 1994) 



Program Records (May, August 1994) 



Program Records (May, August 1994) 



District Records (May, August 1994) 
Graduation File (August 1994) 



Program Records (May, August 1994) 



Program Records (May, August 1994) 



Program Records (May, August 1994) 



TAAS File (May 1994) 
NAPT File (May 1994) 



Program Budget (June, August 1994) 
Program Records (May, August 1994) 
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DISTMBIJTIVE 
INFORMATION 

SYSTEMS 
FOR CAMPUSES 
(DISC) 



MANAGEMENT PLAN 



Contact Person: 
David Wilkinson 



MANAGEMENT CONTEXT 

In a site-based decision making context- 
required in Texas schools by House Bill 2885 
(TEC §21.931) since September 1, 1992~the 
principal and campus leadership team (CLT) 
need timely and comprehensive campus-level 
information for decision making, goal setting, 
planning, budgeting, staffing, instructional 
delivery, assessment, and resource allocation. 
Campus staff need the capability to access 
pertinent data, to manage and analyze them, and 
to create desired reports, independent of central 
processing facilities. At the same time, for 
system-level management and reporting, the 
District needs to maintain the quali y and 
integrity of District data and the efficient 
centralized data processing of major 
applications. 

Begun in 1991-92, Distributive Information 
Systems for Campuses (DISC) is a project to 
increase the data access and information- 
generating capabilities of campuses by 
decentralizing data manipulation functions, 
while maintaining centralized data processing of 
major applications. The primary goal of DISC 
is to "distribute" access and analysis capabilities 
to campuses so that they may use more fully the 
extensive information available on the main- 
frame computer. Distributive information 
systems means that campuses create c sired 
reports more quickly, customize them to meet 
their own needs, and need not rely so much on 
central processing facilities. In effect, DISC is 
about "moving data to the campuses." 

Moving data to the campuses means providing 
campuses with data resident on the mainframe 
computer in a form which they can manipulate. 
In the broadest sense, this effort requires the 
identification of appropriate training for staff, 
hardware, software, and support personnel. 
Because most campuses do not yet have the 
technical capability or the training to create 
their own reports, Phase 1 of the project, 
implemented during the 1991-92 school year, 
concentrated on more immediate objectives, 
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Management 
Context 
Continued 



including identifying software campuses could use, 
furnishing simple applications to campuses (e.g., 
student/parent addresses for directories/ 
mailings), and already-formatted data files for 
schools to manipulate. 

As a transitional stage to campuses creating their 
own custom reports, a second goal of DISC, 
accomplished in Phase 1, was to redesign school 
and District profiles by consolidating a number of 
current reports into a single, comprehensive report 
and, in the process, to create a permanent, on-line 
data file which would be the basis for all future 
profiles. (In the future, schools will be able to 
query downloaded portions of this file.) By the 
end of August 1992, a "Megafile" of school and 
District information was created, two school 
profiles were generated (at the end of February 
1992 and at the end of July 1992), and an 
individual student profile was created with an on- 
line request function. These and other Phase 1 
outcomes are described in Distributive Information 
Systems for Campuses (DISC): 1991-92 Final 
Report (ORE Publication No. 91.24). 

Phase 2 of the DISC project, which began in 1992- 
93, furthered the goals of the project by producing 
District and area profiles, refining profile 
reporting, creating a two-page "executive 
summary 1 * of school profile information, creating a 
user-driven survey process, creating school 
achievement graphs for presentations, and 
promoting campus use of database and other 
commercial software (e.g., Works, WordPerfect, 
Lotus 1-2-3, and Fox). 

Some Phase 2 objectives were not accomplished 
fully in 1992-93 and remain for 1993-94 to: 

• Provide data sets to campuses through 
electronic data interchange (EDI), 

• Identify and promote campus use of database 
and other commercial software, 

• Identify and plan for campuses' training and 
support needs, and 

• Coordinate and integrate DISC activities with 
the continued development of AISD's 
Information Systems Architecture (ISA). 
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In 1993-94, because of other priority 
commitments, ORE relinquishes the leadership 
role in furthering these more developmental 
goals of the project to the Department of 
Man -*ement Infonnation (DM1). With 
assistance from Data Services, ORE will 
superintend the refinement and production of 
profile reports and the District's annual 
performance report (APR). ORE will continue 
to focus on the infonnation needs of campuses 
through its area liaisons and will work 
cooperatively with DMI to provide advice and 
assistance as necessary. 



Management 
Context 
Continued 
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What data do campuses and the District need to support their 
operations and improvement? 



Date Needed: June 1994 
MANAGEMENT QUESTIONS INFORMATION SOURCES 



1-1. What official statistics 

about the schools and the 
District have to be 
reported? According to 
what mandate? 

1-2. What statistics about 

students, the schools, and 
the District are currently 
reported? How often? In 
what format? 

1-3. What statistics does the District 

need for system-level planning and 
improvement purposes? 

1-4. What other statistics do school and 

central staff need for campus planning 
and improvem ent purposes? How often? 
In what format? 

1-5. How can the individual, school, area, 
and District profiles be improved? 

1-6. How should "Megafile" be structured to 
facilitate campus access to 
mainframe information? 



Review of State Law and Regulations 
(Spring 1994) 

Review of School Board Policies 
(Spring 1994) 



Review of Profiles (Spring 1994) 
Review of Academic 
Excellence Indicator 
System (AEIS) (Spring 1994) 



• Information Services 

Committee (ISC) Review (Spring 1994) 



Profile User Comments (Ongoing) 
Administrator Survey (March 1994) 
Evaluation Advisory 
Committee (EAC) Review (Spring 1994) 

Profile User Comments (Ongoing) 
EAC Review (Spring 1994) 

ORE/Data Services Review (Spring 1994) 
Campus Advisory Group Review 
(Spring 1994) 
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MANAGEMENT CONTEXT 



ORE's annual dropout report contains detailed 
information concerning dropout rates in AISD. 
Annual and longitudinal rates are reported for 
various cohorts of students, and for the 9-12 and 
7-8 grade ranges. For the first time this year 
there will be an addition to this report: the data 
collected from those identified as being at risk 
will be correlated with subsequent dropouts. The 
report will also include a projection of expected 
dropouts for the following year. 



DROPOUT/ 
AT-RISK STATISTICS 



MANAGEMENT PLAN 



Contact Person: 
Mario Sanchez, Ph.D. 
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Educational 
Issue 
1 



Which descriptions and statistics will help the District design and 
implement dropout prevention and/or recovery programs? 

Date Needed: January 1994 



MANAGEMENT QUESTIONS 



INFORMATION SOURCES 



1-1. How is the dropout rate changing? 

1-2. What has been the history of 
increases/decreases in annual 
dropout rates since 1985-86: 

a) By ethnicity? 

b) By grade? 

c) By sex? 

1-3. What was the annual dropout rate 
at grades 7-8 and 9-12: 

a) By ethnicity? • 

b) By grade? 

c) By sex? 

d) By school? 

1-4. Is AISD's dropout rate better or 
worse than other districts' rates? 



1-5. Which schools are effective in 
lowering their dropout rates? 

1-6. What are the longitudinal dropout 
rates by ethnicity and by sex, for 
students who were first-time ninth 
graders in: 

a) 1986-87? 

b) 1987-88? 

c) 1988-89? 

d) 1989-90? 

e) 1990-91? 

f) 1991-92? 

g) 1992-93? 

h) Fall 1993? 



• Dropout File Pecember/January 1993-94) 

• Dropout File (December/January 1993-94) 



Dropout File (December/January 1993-94) 



• Dropout File (December/January 1993-94) 

• State Dropout Report (December/January 
1993-94) 

• Dropout File (December/January 1993-94) 



Dropout File (December/January 1993-94) 
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Educational 

Issue 1 
Continued 



MANAGEMENT QUESTIONS 



INFORMATION SOURCES 



1-7. What are the longitudinal dropout 
rates, by ethnicity and by sex, for 
students who wert first-time seventh 
graders in: 

a) 1986-87? 

b) 1987-88? 

c) 1988-89? 

d) 1989-90? 

e) 1990-91? 

f) 1991-92? 

g) 1992-93? 

h) Fall 1993? 



• Dropout File 

• Special Education Monitoring 



System (SEMS) File 



• Dropout File 

• Language File 

• SEMS File 



1-8. What is the dropout rate for: 

a) LEP students? 

b) Special education students? 

c) Students from low-income families? 



• Dropout File 

• Language File 
« SEMS File 



1-9. What percentage of dropouts are: 

a) LEP students? 

b) Special education students? 

c) Students from low- income families? 

1-10. Which traits (characteristics) more 
accurately describe those who 
eventually drop out? 

1-11. What percent of students are 
identified as at-risk? 

a) By ethnicity? 

b) By grade? 

c) By sex? 

d) By school? 



• Dropout File 

• Student Master File 



• At-RiskFile 



• At-Risk File 



1-12. How many at-risk students are 
projected to drop out: 

a) By ethnicity? 

b) By grade? 

c) By sex? 

d) By school? 
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ERLC 



EXTENDED 
SCHOOL 
TIME 



EVALUATION PLAN 



Contact Person: 
David Wilkinson 



PROGRAM DESCRIPTION 

In December 1993, the Superintendent and the 
Management Team reviewed a proposed allocation 
of local resources in ORE and requested that some 
evaluation resources be devoted to several proposed 
projects not assigned to any ORE staff, including 
an examination of the year-round education (YRE) 
concept currently implemented on one AISD 
campus, Sanchez Elementary School. (Ortega 
Elementary School is planning a program along 
similar lines to be implemented beginning in 
August 1994.) Adjustments in full-time equivalent 
(PTE) staff allocations were made to assign .25 
FTE staff to this evaluation. 

Beginning in the 1991-92 school year, Sanchez 
Elementary School began implementing YRE on a 
schoolwide basis. The calendar provides the same 
number of school days as other AISD schools 
have, but the educational program is structured into 
60-day blocks with 20-day "intercessions" between 
them. This sequence repeats three times each year, 
thus providing the usual 180 days of school. 
During the intersessions, teachers and students may 
voluntarily return to school for additional 
instruction and special activities. At-risk students 
are a particular target group for supplemental 
instruction. The Aztec Academy was set up during 
the intersessions to help students with mathematics. 
Summer camp activities, e.g., focusing on nature 
and ecology, are also available to students during 
intersessions. 

This evaluation is not per se an evaluation of the 
program at Sanchez, but rather, an examination of 
the central concept of extending school (i.e., 
instructional) time, whether by amending the 
traditional school calendar (sometimes called the 
"agrarian* calendar), or by extending the school 
day or school year. Arguments for extending 
instructional time are predicated on the notion that 
more time devoted to learning will yield higher 
achievement. The -Sanchez program extends 
instructional time by means of the intersessions. 
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Program 
Description 
Continued 



Other benefits claimed for YRE are that it: 

• Reduces overcrowding by permitting the 
education of more students in the same 
amount of space; 

• Saves money by making better use of existing 
space, thereby reducing the number of new 
schools which need to be built (a 
consideration for future bond programs); 

• May increase student learning by distributing 
intensive learning periods and shorter 
vacation periods throughout the year, thereby 
saving review time and lessening regression 
(loss of skills) over the summer; 

• May prevent teacher and student "burnout" 
by giving pupils and teachers more frequent 
vacations and allowing them to return to 
school refreshed and with renewed energy; 

• Offers extended-year employment for school 
staff who work extra days for extra pay; 

• Offers families off-season vacation 
possibilities; 

• Makes the school available to the community 
throughout the year; and 

• May result in decreased vandalism, discipline 
referrals, and absenteeism. 



Challenges for YRE include: 

• Pupils transferring during the year to a 
9-month school; 

• Program continuity for pullout programs 
(e.g., band and orchestra); 

• Parents having to plan activities (e.g., child 
care, summer camp, and recreation) within a 
society geared for 9-month schools; 

• Family planning when some family members 
attend 9-month schools; 

• Pupils possibly not being on vacation at the 
same time as their friends; 

• Scheduling for university courses (e.g., for 
staff continuing their educations); and 

• Communicating with the community. 



Because only a quarter-time person can be 
dedicated to this evaluation, the evaluation 
outlined below is primarily descriptive. A 
comprehensive study, or series of studies, would 
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be required to provide a definitive answer to the 
question of whether a year-round school program 
or any modification of the customary September -to- 
May school calendar which provides more 
instructional time is "better" than the traditional 
program. Some outcome information will be 
provided through ORE's generic evaluation system 
(GENESYS), but inferences which may be drawn 
from this information will be limited. 



Program 
Description 
Continued 
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Decision 
Question 
1 



Should District schools continue to be offered the opportunity to change 
from the traditional school calendar to year-round education (YRE)? 



EVALUATION QUESTIONS 



1-1. What does the research literature 

indicate about the effective use of extended 
school time? 

1-2. What activities took place during the 
intersessions during 1993-94? 

1-3. Which staff provided supplemental 
instruction? 

1-4. Kow many students participated in 
intersession activities at Sanchez 
during the 1993-94 school year? 

1-5. How did the students attending intersession 
activities at Sanchez during 1993-94 
compare to the students not attending 
them, in terms of: 

• Achievement? 

• Attendance? 

• Discipline rates? 

• Retention rates? 

1-6. What are the costs of extending school 
time in relation to instructional gains? 



Date Needed: July 1994 

INFORMATION SOURCES 
• Literature Review (Spring 1994) 



• Program Records (Spring 1994) 



Program Records (Spring 1994) 



Program Records (Spring 1994) 



• GENeric Evaluation SYStem (GENESYS) 
(Summer 1994) 



• Program Budget (Spring 1994) 

• Program Records (Spring 1994) 

• GENESYS (Summer 1994) 
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MANAGEMENT CONTEXT 



GENERIC EVALUATION 
SYSTEM 
(GENESYS) 



MANAGEMENT PLAN 



Contact Persons: 
Mario Sanchez, Ph.D. 
David Wilkinson 



GENESYS is a GENeric Evaluation SYStem. 

GENESYS is a method of streamlining data 
collection and evaluation through use of computer 
technology. From its first year in 1973, the Office 
of Research and Evaluation (ORE) ha-, been 
challenged to evaluate a multitude of diverse 
programs with limited resources. By standardizing 
methods and information provided to persons 
requesting data, GENESYS makes it possible to 
evaluate a much larger number and variety of 
programs than would ordinarily be possible. 

GENESYS gathers and reports the following 
standard infora ation on specified groups of 
students: 



Student characteristics, 
Achievement, 
Attendance, 
Discipline, 
Grades/credits, 
Dropout status, 
Retention status, and 
At-risk status. 



GENESYS can be run for any group identifiable 
through a computer file. Since 1988-89, the first 
year in which it was run, GENESYS has included 
a wide variety of elementary, secondary, and K-12 
programs. The programs likely to be included in 
1993-94 are listed at the end of this plan. 
Additional groups will probably be identified 
during the course of the year for which information 
will be required. 

Given a file of those students involved in a 
program, group, or innovation, GENESYS 
provides outcome information for the following 
variables: 

Group Characteristics: Number served by 
grade, ethnicity, sex, low income, limited- 
English-proficient, overage for grade, at- 
risk, special education, and gifted/talented; 
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Achievement Results by Grade: Performance 
on the ITBS, TAP, TAAS, and the Report 
on Program Effectiveness (ROPE) regression 
trend information; 

Attendance, Discipline, Grades/Credits: 
Four semesters; 

Dropouts and Retainees: Counts as of the 
end of the fifth six weeks for dropouts and 
potential retainees as of the end of May. 

For each group, four standard listings are 
produced: 

The GENESYS Program Summary summarizes 
information on the group's overall performance 
on all variables. 



Management 
Context 
Continued 



The Executive Summary summarizes findings in 
narrative form and compares the group's data to 
appropriate districtwide groups. 

GENESYS Data by Student provides a listing of 
this information by individual student (as 
applicable) for review and -reference. 

The Program Description, supplied by program 
or evaluation staff, gives information on the 
program's characteristics. 

Three optional printouts can be requested by the 
user: 

Cross-Program Comparison Charts provide a 
summary of statistics across multiple programs 
designated by the user; 

Two-Way Crosstabulation Tables provide a 
greater level of detail about selected variables 
than that provided in the evaluation summary. 

Individual ROPE Residuals provide detailed 
ROPE scores for individual students. 

Information sources for all of the following 
questions are the computer files accessed by 
GENESYS. 
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Educational 
Issue 
1 



What do outcome data from AISD computer files indicate about the 
status of students in special programs or other identifiable groups? 



Date Needed: June 1994 



1-2. 



1-3. 



MANAGEMENT QUESTIONS 

1-1. What were the demographic 
characteristics (number served, 
ethnicity, sex, etc.) of the 
students in the special programs 
or identified groups? 

What were the attendance rates for 
students in the special program or 
identified groups? 

How many discipline incidents 
occurred among students in the 
special programs or identified 
groups? 

1-4. What were the grade point 

averages (GPA's) earned by the 
students in the special programs 
or identified groups? 

1-5. How many course credits were 
earned by the students in the 
special programs or identified 
groups? 

1-6. How many F*s and NG's (no 
grades) were received by the 
students in the special 
programs or identified groups? 

1-7. What were the dropout rates for 
students in the special programs 
or identified groups? 



INFORMATION SOURCES 
• Computer Files 



• Computer Files 



Computer Files 



• Computer Files 



Computer Files 



Computer Files 



Computer Files 
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Educational 

Issue I 
Continued 



1-8. 



1-9. 



1-10. 



1-11. 



MANAGEMENT QUESTIONS 

How many students in the special 
programs or identified groups 
were retained? 

What were the achievement levels 
of the students in the special 
programs or identified groups? 

What was the at-risk rate for 
students in special programs or 
identified groups? 

What was the graduation rate for 
students in special programs or 
identified groups? 



INFORMATION SOURCES 

• Computer Files 

• Computer Files 

• Computer Files 

• Computer Files 
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GENESYS GROUPS 

A generic evaluation is projected for each of the following programs or groups at the conclusion of the 
1993-94 school year. Additional programs may be identified during the course of the year. 

A. AIM High, 1993-94 

AISD Elementary School Students, 1993-94 

AISD Junior High Students, 1993-94 

AISD Senior High Students, 1993-94 

AISD Students in Grade 9, 1993-94 

(AISD) Students Served by At-Risk Counselors, Fall 1993 

(AISD) Students Served by At-Risk Counselors, Spring 1994 

(AISD) Students Served by At-Risk Counselors, 1993-94 

(AISD) Students Trained in Conflict Resolution, 1993-94 

Alternative Learning Center (ALC), Grades 7-8, 1993-94 

Alternative Learning Center (ALC), Grades 9-12, 1993-94 

Austin Youth River Watch Program, 1993-94 Trainees 

B. Believe in Me, 1993-94 
Block Program at LBJ, 1993-94 
Block Program at Lanier, 1993-94 
Block Program at Reagan, 1993-94 
Block Program at Travis, 1993-94 
Business School Partnership (LBJ), 1993-94 

C. Campus-Based DFS Programs - Elementary, 1993-94 
Campus-Based DFS Programs - Middle/Junior High, 1993-94 
Campus-Based DFS Programs - High School, 1993-94 

Club ESOS, Middle School, 93-94' 

Club ESOS, High School, 93-94 

Community-Based Educational Aternatives, 1993-94 

Content Mastery, Ch. 1 Students, 1993-94 

Coordinated Vocational Academic Education, Grades 6-8, 1993-94 

Coordinated Vocational Academic Education, Grades 9-12, 93-94 

D. DARE Participants - 1993-94 - 5th Grade 
DARE Participants - 1993-94 - 7th Grade 
DFS Participants - Elementary - 1993-94 
DFS Participants - Middle School - 1993-94 
DFS Participants - High School - 1993-94 

E. Education for Parenthood Centers (Johnston), 1993-94 
ESOS Academy, 1993-94 

Evening School, 1993-94 

Extended Day Program, Ch. 1 Schools, 1993-94 

F. None 
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G. 



None 



H. High School Block Programs, 1993-94 
Hispanic Student Initiative Scholarhip, 1993-94 
1993-94 HOSTS Chapter 1 Students (Dawson) 
1993-94 HOSTS Chapter 1 Students (Ortega) 
1993-94 HOSTS Chapter 1 Students (Zavala) 
1993-94 HOSTS - Ortega, Zavala, and Dawson 
HOTS, Chapter 1 Students, 1993-94 

I. 1993-94 Intersession Tutoring Program (Sanchez) 

I. Johnston Learning Technology Center, 1993-94 
Jumpstart (McCallum), 1993-94 

K. Kealing Magnet, 1993-94 

L. LBJ Non-Science Academy Students, 1993-94 

LEP A & B High School Students Not in Title VII 

LEP Denials, Elementary, 1993-94 

LEP Denials, Middle School, 1993-94 

LEP Denials, Elementary, 1993-94 

LEP Served, Elementary, 1993-94 

LEP Served, Middle School, 1993-94 

LEP Served, Elementary, 1993-94 

Liberal Arts Academy, 1993-94 

M. Megaskills 1993-94, Elem. School Children of Participants 
Megaskills 1993-94, Jr. High School Children of Participants 
Megaskills 1993-94, High School Children of Participants 

N. None 

O. Overage 9th-Grade Student Programs 

P. PALees - 1993-94, Elementary 
PALees - 1993-94, Middle School 
PALees - 1993-94, High School 
Pathways, Experimental, 1993-94 
Pathways, Control, 1993-94 
PEAK, 1993-94 

Project HELP, Elementary, 1993-94 

Project HELP, Middle School, 1993-94 

Project HELP, Senior High, 1993-94 

Project MAN (LBJ), 1993-94 

Project Mentor, Elementary, 1993-94 

Project Mentor, Middle/Junior High School, 1993-94 

Project Mentor, High School, 1993-94 
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Q. None 

R. 1993-94 Reading Recovery, Chapter 1 Students 
Robbins Secondary School, 1993-94 

S. SABES/SESOS, Ch. 1 Schools, 1993-94 
St. Edward's Mentor Program, 1993-94 
Science Academy, 1993-94 
Secondary Honors Program, 1993-94 
Summer School, 1993-94 
Sunshine Camp, 1993-94 
Student Retreat Program, Elementary, 1993-94 
Student Retreat Program, Middle School, 1993-94 
Student Retreat Program, High School, 1993-94 

T. Title VII Newcomers Program, 1990-91 
Tide VII Newcomers, 1991-92 
Title VII Newcomers, 1992-93 
Tide VII Newcomers Program, 1993-94 
Transitional Bilingual Education (Martin), 1993-94 

U. None 
V. None 

W. Writing to Read, Ch. 1 Students, 1993-94 
Writing to Write, Ch. 1 Students, 1993-94 

X. None 

Y. Year-Round Program (Sanchez), 1993-94 

Z. Zenith (Evening School), 1993-94 
Zenith Program, 1993-94 
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FORMER 
STUDENT 
SURVEY 



EVALUATION PLAN 



Contact Person: 
Natalie Rodgers 



I 
I 
I 
I 
I 
I 
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ERIC 



PROGRAM DESCRIPTION 

In September of 1991, a survey of 1989-90 AISD 
high school graduates (the class of '91) was 
conducted. This survey of former students, 
which was last administered to 1981-82 
graduates, was conducted in response to AISD's 
fourth strategic objective, "After exiting AISD, 
all individuals will be able to perform 
successfully at their next endeavor." 

The Former Student Survey asked questions of a 
sample of graduates in three main areas: 

• What students are doing now, 

• What high school courses students found most 
useful, and 

• How prepared students felt for their present 
activities. 

In 1992, the survey was redesigned based on 
review by District staff and input from 
community agencies. Several items in the revised 
survey were linked to the goals for graduates 
outlined in the Project A+ report Children of 
Promise: Educating for the Future. The intent 
of this year's survey will be to continue assessing 
the perception of former AISD high school 
graduates about their education, employment, and 
community service activities. 
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Should AISD programs, policies, and procedures be modified to prepare high school 
graduates better for postgraduation activities? 



Date needed: December 1994 



EVALUATION QUESTIONS 



INFORMATION SOURCES 



1-1 . What are the opinions of former AISD high 
school graduates on how well their high 
school education prepared them for: 

• Continuing education, 

• Employment, and 

• Community service activities? 



• Former Student Survey 
(December 1993) 



1-2. How should the former student survey be 
modified to reflect current job 
classifications and changing roles in 
today's society? 



• Community Agencies 

• Advisory Groups 



District Staff 
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HISTORICAL STUDENT 
RETENTION IN GRADE 



MANAGEMENT PLAN 



Contact Person: 
Mario Sanchez, Ph.D. 



MANAGEMENT CONTEXT 

Student retention in grade is documented annually 
in the historical retention report. This report 
contains data concerning the number of students 
who repeat a grade level. The report is a 
compilation of retention statistics over time. The 
report is produced once a year, although the 
information that goes into it is gathered at two 
points through the year, once after the end of the 
school year (potential retainees are counted), and 
once after the beginning of the next school year 
(actual retainees are counted). The most recent 
analysis, in the form of tables and graphics, is 
included in the report. This analysis uses the 
most recent data as well as longitudinal retention 
statistics. 
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Educational 
Issue 
1 



What questions could be addressed using District computer files that 
could indicate trends and underline policies concerning the retention of 
AISD students? 



MANAGEMENT QUESTIONS 

1-1. What are the most recent retention rates 
given by grade and by particular grade 
span? 

1-2. How do the most recent retention results 
compare with past years? 

1-3. What are the present and past statistics 
concerning end-of-year potential retainees 
as a percentage of: 

a) Average daily membership for the 
year? 

b) End-of-year membership? 

1-4. What are the present and past statistics 
concerning beginning-of-year actual 
retainees as a percentage of: 

a) Average daily membership for 
the previous year? 

b) End-of-year membership for 
the previous year? 

c) Fall enrollment? 

d) Previously identified potential 
retainees? 



Date Needed: September 1994 

INFORMATION SOURCES 



• Student Master File 

• Average Daily Membership File 
(November) 

• Student Master File 



• Student Master File 

• Elementary Potential Retainee File 

• Student Grade Reporting File 
(July) 



• Student Master File 

• Average Daily Membership File 
(November) 

• Elementary Potential Retainee File 



1-5, How many LEP students were retained in 
1993-94 by grade level? 



1-6. What changes occurred in the rate of 
retention from last year to this? In what 
direction and in which grades did these 
changes occur? 



• Elementary Potential Retainee File 

• Student Grade Reporting File 
(July) 

• LEP File 

• Student Master File (July) 
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OVERAGE 9TH-GRADE 
STUDENT PROGRAMS 



EVALUATION PLAN 



Contact Person: 
David Wilkinson 



PROGRAM DESCRIPTION 

One of the five priority areas in the 1993-94 District 
Improvement Plan is student outcomes in grades 6-9. 
Objective 2 for this improvement area is, "Reduce the 
numbers of overage grade 9 students/ Accordingly, 
in the District's 1993-94 budget, each of the four 
operations areas was allocated $100,00 to be used for 
this purpose. The Superintendent specified that most 
of the funds in each area were to be used at one 
school in order to focus the program and have a 
higher chance of success. This direction was followed 
except in Area 4 and with a slight deviation in Area 1. 
The table on the following page contains an overview 
of the programs for each area. More extensive pro- 
gram descriptions were transmitted to the Board in the 
Superintendent's Board Update of September 3, 1993. 

Outcome evaluation for each of the programs will be 
conducted by the Office of Research and Evaluation 
(ORE) through the use of its GENeric Evaluation 
SYStem (GENESYS). Some limited process 
information will be collected via an ORE survey of 
school staff. Formative evaluation (e.g., using student 
portfolios) will be conducted by the program managers 
at some program sites. GENESYS is a method of 
streamlining data collection and evaluation through the 
use of computer technology. By standardizing input 
data and outcome information, GENESYS makes it 
possible to evaluate a large number and variety of 
programs with limited resources. GENESYS has been 
employed successfully in the evaluation of dropout 
prevention programs since 1988-89. In 1992-93, 
GENESYS information made possible cost- 
effectiveness comparisons of many dropout prevention 
programs. GENESYS gathers information from 
central computer files and reports information, for any 
specified group of students, on: 



Student demographic characteristics, 

Achievement, 

Attendance, 

Discipline, 

Grades/credits, 

Dropout status, 

Retention status, and 

At-risk status. 
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Program 
Description 
Continued 



GENESYS compares the performance of the specified 
group with District and, for some variables, national 
performance. A Report on Program Effectiveness 
(ROPE) analysis in GENESYS compares predicted 
with actual student achievement to determine if there 
are significant differences because of participation in a 
program. A related GENESYS analysis compares the 
predicted number of dropouts in a group to the actual 
number. 

ORE conducts an annual survey of campus 
professional and^administrative staff. Information 
regarding progfam implementation and staff opinion 
about the programs for overage ninth graders will be 
obtained. Cost-effectiveness comparisons among 
programs will be made. 
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OVERVIEW OF AREA PROGRAMS FOR OVERAGE 9TH-GRADE STUDENTS 



Area 


Program Site 


Program Name 


Main Program Features 


1 


Reagan High School 


Project AVID 
(Advancement Via 
Individual 
Determination) 


—Support team for stucents of two teachers, 
a social worker, a counselor, and an 
administrator 

—Successful teachers assigned to program 
students 

—Tutorial class with computers 

—Parent conferences 

—Staff training in cooperative learning 

strategies and performance-based assessment 


2 


Johnston High School 


Partnerships Activated 
for School Success 
(PASS) 


—Faculty training in Total Quality Learning 

strategies; student training by faculty 

—Faculty training in accelerated learning 

instructional strategies 

—Vertical team linkages among students and 

staff (mentoring, peer tutoring, other peer 

support strategies) 

—Setting quality standards for the 

instructional, counseling, and career 

guidance programs 


3 


Travis High School 


Not named 


—Interdisciplinary team organization 
-Support staff of teachers, parent 
intervention specialist, and social worker 
—Integrated learning system on IBM network 
—Adult mentors and tutors 
-Referral to GED programs for some 
students 


4 


Bowie High School 


School-to-Work 
Transition Program 


—Interdisciplinary program, centered around 
a core curriculum 
—Private sector involvement 
—On-the-job training; apprenticeship 
—Company funding of student postgraduate 
study 


4 


LBJ High School 


Health Services 
Academy 


-Student research into "hot" health topics 

—Student dissemination of information 

through pamphlets and television programs 

—Student development of long-range school- 

to-work nlan 

-Four parent workshops 

—Monetary incentives for students and 

parents 

—Student field trips to area 
colleges/universities; one extended trip 
—Staff mentor for students 
—Student interactions with community 
resource persons 



T39 



93.07 



Decision 
Question 
1 



Should each of the area programs for overage 9th-grade students be 
continued as is, modified, or discontinued? 



EVALUATION QUESTIONS 

1-1. What were the demographic 
characteristics (number served, 
ethnicity, sex, etc.) of the 
students in each of the area 
programs? 

1-2. What were the attendance rates for 
students in each of the area 
programs? 

1-3. How many discipline incidents 

occurred among students in the area 
programs? 

1-4. WTiat were the grade point averages 
(CPAs) earned by the students in the 
area programs? 

1-5. How many course credits were earned 
by the students in the area programs? 

1-6. How many F's and NG's (no grades) 
were received by the students in the 
area programs? 

1-7. What were the achievement levels of 
the students in the area programs? 

1-8. What were the dropout rates for 
students in the area programs? 

1-9. How many students in each of the 
area programs were retained? 



Date Needed: June 1994 

INFORMATION SOURCES 



• GENeric Evaluation SYStem 
GENESYS (Summer 1994) 



GENESYS 



GENESYS 



GENESYS 



GENESYS 



GENESYS 



GENESYS 



GENESYS 



GENESYS 



ERLC 



P-4 



140 



93.07 



Decision 
Question 1 
Continued 



EVALUATION QUESTIONS 

1-10. How many students in each of the area 
programs left school to pursue a GED? 
How many students obtained GEDs? 

1-11. What did principals and teachers think of 
each area program? What components 
were identified as successful? 

1-12. What did each of the area programs cost 
per student? 



INFORMATION SOURCES 

• Program Records (Spring 1994) 

• School Survey (March 1994) 

• Program Budget (April 1994) 



PS 
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PROGRAM 
EFFECTIVENESS 
COMPARISONS 



MANAGEMENT PLAN 



Contact Person: 
David Wilkinson 




MANAGEMENT CONTEXT 

Each year, ORE reports to the Board of Trustees 
on the effectiveness of special programs in the 
District. ORE prepares program effectiveness 
charts for the Board's annual budget study session 
in February. Until 1992-93, ratings of program 
effectiveness were informed judgements based on 
evaluation findings and knowledge of the 
projects. In 1991, the Board asked ORE to 
provide it with a measure of effect as well as 
cost. ORE responded during the 1992-93 school 
year with a retrospective look at 1991-92 AISD 
programs. In February 1993, ORE presented the 
Board with program effectiveness charts which 
included cost-effectiveness ratios for many 
programs evaluated during 1991-92. 

ORE staff worked during fall 1992 to develop the 
program effectiveness comparisons process. Staff 
addressed an assortment of issues associated with 
cost-effectiveness comparisons, such as how costs 
should be calculated, how effect sizes should be 
determined, how different programs can be 
compared according to a common scale, and how 
the information should be reported. Internal 
Audit staff and the Evaluation Advisory 
Committee (EAC), which includes principals, 
teachers, parents, and representatives of the 
general public, were consulted as the comparison 
process was developed. Draft comparison charts 
were sent to program staff for review, and a 
number of changes were subsequently made based 
on the comments received. The comparison 
process was also presented for professional 
scrutiny at the annual meeting of the Texas 
Research and Evaluation Network (TREN). 
Finally, staff presented the draft results to the 
School Board to secure review and comments 
from trustees. A final report was published in 
May 1993. 

The methodology developed in fall 1992 for 
assessing the cost-effectiveness of programs was 
applied to the programs evaluated by ORE in 
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1992-93, and cost-effectiveness information 
included in ORE's 1992-93 final evaluation 
reports. Program effectiveness comparisons for 

1992- 93 programs will be reported to the Board 
of Trustees in February 1993 when the Board 
begins its deliberations on the District's budget 
for 1994-95. During the 1993-94 school year, 
ORE staff will refine the methodology for 
calculating cost-effectiveness and apply it to 

1993- 94 programs being evaluated. 



Management 
Context 
Continued 
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What features should be included in program effectiveness 
comparisons? 



1-1. 



1-2. 



1-3. 



1-4. 



1-5. 



Date Needed: May 1994 
MANAGEMENT QUESTIONS INFORMATION SOURCES 



What critical information about programs 
do decision-makers need to be able 
to make effective changes? 



What cost-effectiveness information 
should be reported? How often? 
In what format? 



What methodologies should be employed 
to assess: 

a) Cost? 

b) Effect size? Review? 

c) Cost/effect ratio? 

d) Dropout prevention effect? 

e) Drug abuse prevention effect? 

How can the cost-effectiveness 
methodology be improved? 



How can the cost-effectiveness 
information currendy being reported 
about programs be expanded beyond 
programs (e.g., to schools)? 



Management Team Review (February 1994) 
School Board Review (February 1994) 
Evaluation Advisory Committee 
(EAC) Review (February 1994) 

Review of Calendar of Information and 
Decision Events (December 1993) 
Management Team Review (February 1994) 
School Board Review (February 1994) 
EAC Review (February 1994) 

Literature Review (February 1994) 
Program Staff Review (February 1994) 
EAC Review (February 1994) 
Management Team Review (February 1994) 
School Board Review (February 1994) 



Literature Review (February 1994) 
Collegial Review (Conference 
Presentation) (April 1994) 
Program Staff Review (February 1994) 
EAC Review (February 1994) 

Management Team Review (February 1994) 
EAC Review (February 1994) 
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PROGRAMS FOR 
LIMITED-ENGLISH- 
PROFICIENT 
(LEP) STUDENTS 



EVALUATION PLAN 



Contact Person: 
David Wilkinson 



PROGRAM DESCRIPTION 

Texas law requires that all students with a 
language other than English (LOTE) be processed 
to determine their English proficiency. Those 
identified as limited English proficient (LEP) 
must be provided one of two basic programs: 

• Bilingual Education (BE), a transitional 
program of dual-language instruction 
including instruction in the home language 
and English as a Second Language (ESL) for 
a minimum of 45 minutes daily, provided to 
students in any language classification for 
which there are 20 or more students enrolled 
in the same grade level, and 

• English as a Second Language (ESL), a 
program of specialized instruction in English 
provided to students not receiving bilingual 
education and to students whose parents 
refuse dual-language instruction. (Parents 
may also refuse ESL instruction.) 

Some LEP students are served by Special 
Education only. 

A total of ,30 full-time equivalent (FTE) staff will 
be devoted to monitoring the progress of LEP 
students, especially in terms of English 
achievement. As resources allow, management 
information will also be provided through the 
computerized file of all LOTE students in the 
District. This file, the LEP master file 
(sometimes called the LANG file), will be 
redesigned this year to incorporate test 
information from the Prueba de Realizacfon and 
to ensure that all important information about 
LEP students is maintained and accessible to 
users. 

Other evaluation plans including questions related 
to LEP students are Chapter 1, Chapter 1 
Migrant, Dropout Prevention, Historical Student 
Retention in Grade, Title VII, and Student 
Achievement. Evaluation findings about LEP 
students will be presented in the final evaluation 
reports for these projects. 
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Are limited-English-proficient (LEP) students receiving an instructional 
program appropriate to their needs? 



Date Needed: July 1994 



EVALUATION QUESTIONS 



INFORMATION SOURCES 



Findings from Evaluation Questions 1-1 through 7-J will be reported in the final report on student 
achievement for 1993-94. 



1-1. What percentage of AISD LEP 

students were tested in English on the 
TAAS, by grade? 

1-2. Whatpercer' ?e of AISD LEP 
students teswd in English mastered 
TAAS minimum requirements, by 
subject (mathematics, reading, 
writing, and all tests taken) and grade: 

(a) Compared to the previous year? 

(b) Compared to the State average? 

1-3. What were the characteristics of the 
Spanish-speaking LEP students tested 
with the ITBS or NAPT (years in AISD, 
language dominance, program 
received, etc.)? Of LEP students 
speaking other languages? 

1-4. What were the mean grade equivalent 
(GE) achievement gains of LEP 
students tested with the ITBS or 
NAPT in each of the last two years, 
by grade and test area and by language 
(Spanish and other)? 

1-5. How did the achievement of LEP 
students tested in Spanish on the 
Prueba de Realizacfon in 1993-94 
compare with that of the LEP students 
tested in 1992-93, by grade and test 
area? 



Texas Assessment of Academic 
Skills (TAAS) File (May 1994) 



TAAS File (May 1994) 



LEP File (June 1994) 

Iowa Tests of Basic 

Skills (ITBS) File (May 1994) 

Norm-referenced Achievement 

Program for Texas (NAPT) File (May 1994) 



LEP File (June 1994) 
ITBS File (May 1994) 
NAPT File (May 1994) 



LEP File (June 1994) 
Prueba de Realizacfon 
File (June 1994) 
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Key 
Issue 1 
Continued 



Findings from Evaluation Question 1-6 will be reported in the final report on retention for 1993-94. 
EVALUATION QUESTIONS INFORMATION SOURCES 



1-6. How many LEP students were potential 
retainees at the end of the 1992-93 school 
year, by grade level? How many LEP 
students who were potential retainees at 
the end of the 1992-93 school year were 
actually retained in the same grade level 
in 1993-94? 

1-7. How did the LEP students being served 
compare to the LEP students denied 
service by their parents, in terms of: 

• Achievement? 

• Attendance? 

• Discipline rates? 

• Grades? 

• Retention rates? 

• Dropout rates? 

1-8. What is the nature of AISD's LEP 
population: 



By grade? 

By language? 

By language dominance? 

By parent denial status? 

By program received? 

By age? 

By special education status? 
By low-income status? 
By gifted/talented status? 
By discipline status? 
By attendance? 
By dropout status? 
By length of service? 



• Retention File (June 1994) 

• Student Grade Reporting 
(SGR) File (June 1994) 



• Office of Research and 

Evaluation GENtric Evaluation 
SYStem (GENESYS) (Summer 1994) 



• LEP File (June 1994) 

• GENESYS (Summer 1994) 
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Key 
Issue 1 
Continued 



EVALUATION QUESTIONS 



INFORMATION SOURCES 



1-9. In 1993-94, how many LEP students: 

a) Have progressed from one language 

level tO another? (Annual reporting 
required by State law) 

b) Have been exited from the bilingual 
education and English as a second 

language program? (Annual reporting 
required by State law) 

c) Graduated? 

1-10. ' How many LEP students transferred 
from their home campuses to other 
campuses to obtain additional 
services? 



LEP File (June 1994) 
Graduate File (June 1994) 



LEP File (June 1994) 



1-11. How many teachers and aides have 

been trained? (Annual reporting required by 
State law) 

1-12. How much did the programs cost per 
student? 



• Employee Master File (May 1994) 

• Program Records (May 1994) 

• LEP File (June 1994) 

• General Ledger Master File (June 1994) 



Findings from Evaluation Question 1-13 will be reported in the final report on AISD dropouisfor 1993-94. 



1-13. How many LEP students dropped out? 
Compared to other AISD students? 
How many LEP students are identified 
as "high risk H ? 



• LEP File (June 1994) 

• Annual Dropout File (June 1994) 

• Student Master File (SMF) (June 1994) 



Date needed: December 1994 
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Does the LEP master file contain and have accessible all important 
information about LEP students needed for official reporting and 
planning purposes? 



Date Needed: August 1994 

EVALUATION QUESTIONS INFORMATION SOURCES 

2-1. What information should the LEP File • Review by Program and Evaluation 

contain? What information should be Staff (March 1994K 

displayed on screen? In what format? ' 
Who should have access to the 
information in the file? By what 
means? 
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How effective is prekindergarten service in facilitating successful 
outcomes for LEP students? 



Date Needed: August 1994 



EVALUATION QUESTIONS 



INFORMATION SOURCES 



3-1. How did LEP students who were served 
in prekindergarten compare to students 
who were not served in pre-K, after five 
years of service, in terms of: 



• GENESYS (Summer 1994) 

• LEP File (June 1994) 



Achievement? 
Attendance? 
Discipline rates? 
Grades? 
Exit rates? 
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STUDENT 
ACHIEVEMENT 



EVALUATION PLAN 



Contact Persons: 
EvangelinaMangino, Ph.D. 
Natalie Rodgers 
Barbara Wiser 



I 
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PROGRAM DESCRIPTION 

The Systemwide Testing Program coordinates the 
administration and processing of achievement 
tests and the distribution of their results. The 
program is also designed to assist AISD adminis- 
trators, teachers, students, and parents in the use 
of achievement data in the following areas: 

• To investigate the overall impact of AISD's 
programs on majority and minority student / 
achievement at the District and campus levels, 

• To identify students eligible for remedial and 
enrichment programs, as well as different 
levels of instruction within the regular 
instructional programs, 

• To provide achievement data used for the 
evaluation of remedial and enrichment 
programs, as well as the regular instructional 
programs, 

• To report basic information on student 
achievement to the School Board and the 
general public, 

• To maintain Texas Assessment of Academic 
Skills (TAAS) and Texas Educational 
Assessment of Minimum Skills (TEAMS) 
exit-level files to determine students' status 
for high school graduation, 

• To identify and respond to achievement- 
related information and resource needs of the 
School Board, campus and central office 
administrators, teachers, counselors, and 
parents, and 

• To provide support to campus and central 
administrators and teachers to utilize 
achievement data, 

• To identify schools performing higher and 
lower than schools with similar populations. 

In addition to the coordination of tests and report 
of their results, the systemwide testing program 
participates in the following activities: 

• Assisting TEA with field tests as required by 
state law. 
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Decision 
Question 
1 



Is AISD's instructional program successfully teaching students the 
skills required to pass the TAAS objectives? 



Date needed: July 1994 



EVALUATION QUESTIONS 

1-1. What percentage of students passed the 
TAAS test in mathematics and reading hi 
grades 3-8, and writing, social studies, 
and science in grades 4 and 8 at the 
"minimum expectation" and "mastered all 
objectives" levels: 

a) By sex? 

b) By ethnicity? 

c) By low-income status? 

d) By LEP status? 

1-2. How does the achievement performance 
in AISD compare to the State 
performance and other urban districts? 



1-3. How many schools met the required gain 
to achieve the State standard in three 
years (90 percent passing): 

a) All students? 

b) Low-income students? 

1-4. How did AISD students (total and by 

ethnicity, income status, and LEP status) 
perform on the TAAS, grades 3-8: 

a) Compared to last year? 

b) Compared to other urban districts? 

c) Compared to other Texas districts? 

d) Compared to State averages? 

1-5. What percentage of AISD LEP and 

special education students were tested on 
the TAAS? 

1-6. How many schools were identified as low 
achieving campuses by TEA and how 
does this compare with: 

a) Other urban districts? 

b) Previous years? 



INFORMATION SOURCES 

• Texas Assessment of Academic Skills 
(TAAS) File (December 1993) 

(June, August 1994) 



TAAS File (December 1993) 

(June, August 1994) 
TAAS State Report (July 1994) 
JUEC/TAAS Summary Reports (June 1994) 

TAAS File (December 1993) 

(June, August 1994) 



TAAS File (December 1993) 

(June, August 1994) 
Joint Urban Evaluation Council (JUEC) 
TAAS Summary (June 1994) 



TAAS File (December 1993) 

(June, August 1994) 



TEA List of Low-Achieving Campuses 
(August 1994) 
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Decision 
Question 
2 



Is AISD's instructional program successfully teaching students the 
skills required to pass the Exit-Level TAAS objectives? 



Date needed: July 1994 



EVALUATION QUESTIONS 

2-1. How many and what percentage of the 
seniors with all other requirements for 
graduation completed were denied a 
diploma as a result of not mastering the 
Exit-Level TAAS? 

2-2. How many and what percentage of tenth- 
grade students met the mastery cn ria 
on the Exit-Level TAAS: 

a) At the first administration? 

b) At the retesting? 

c) Overall? 

2-3. How did tenth graders perform on the 
Exit-Level TAAS: 

a) By sex? 

b) By ethnicity? 

c) By low-income status? 

d) By LEP status? 

2-4. How did tenth graders perform on the 
writing sample? 

2-5. How did AISD students (total and by 

ethnicity, income status, and LEP status) 
perform on the Exit-Level TAAS: 

a) Compared to last year? 

b) Compared to other urban districts? 

c) Compared to other Texas school 
districts? 

d) Compared to the state averages? 

2-6. How did students perform at each 
campus? 



INFORMATION SOURCES 

• TAAS File (December 1993) 

(June, August 1994) 

• Graduate File (June 1994) 



TAAS File (December 1993) 

(June, August 1994) 



• TAAS File (December 1993) 

(June, August 1994) 



TAAS File (December 1^93) 

(June, August 1994) 

TAAS File (December 1993) 

(June, August 1994) 

Texas Education Agency (TEA) 

Data Tapes (December 1993) 

(June, August 1994) 

TAAS State Report (July 1994) 



TAAS File (December 1993) 

(June, August 1994) 
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Decision 
Question 
3 



Is AISD's instructional program successfully teaching students the 
skills required to compete successfully with students nationwide? 



Date needed: June 1994 



EVALUATION QUESTIONS 



INFORMATION SOURCES 



3-1. How did AISD's 1993-94 student 

achievement compare, by ethnicity, to 
the students: 

a) In the nationwide norming sample? 

b) Compared to other urban districts? 

c) In Texas? 

3-2. What percentage of the students in each 
ethnic group scored in the highest and 
lowest ranges of the NAPT and ITBS? 

3-3. How large were the achievement gains 
made by AISD students in 1993-94 by 
ethnicity? 

3-4. How did AISD's 1993-94 student 

achievement compare, by ethnicity, to 
the achievement of students in grades 
1-11 across the last two years? 

3-5. What were the achievement levels of 
AISD students, by ethnicity, who 
qualified for a free or reduced-price 
meal: 

a) In 1993-94? 

b) Compared to the achievement of 
low-income students from previous 
years? 

3-6. Which schools showed achievement in 

reading and mathematics higher or lower 
than schools with similar populations? 

3-7. What are the mean grade equivalent (GE) 
achievement gains of LEP students able 
to be tested in English in 1993-94? 



Norm-referenced Assessment Program for 
Texas (NAPT)/Iowa Tests of Basic Skills 
(ITBS) file (June 1994) 



NAPT/ITBS File (June 1994) 



NAPT/ITBS File (June 1994) 



NAPT/ITBS File (June 1994) 



NAPT/ITBS File (June 1994) 



NAPT/ITBS File (June 1994) 

Report on School Effectiveness (ROSE) 

(July 1994) 

NAPT/ITBS File (June 1994) 
LANG File (Ongoing) 
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Information 
Needs 



Date needed: July 1994 



INFORMATION QUESTIONS 

1-1. How did AISD's 1993-94 graduates who 
took the SAT compare with other 
students nationwide: 

a) In reading and mathematics? 

b) In participation rates? 

c) In previous years? 

1-2. What is the correlation of passing the 
end-of-course tests with passing the 
course in Algebra I or Biology I? 

1-3. What percentage of AISD students passed 
the end-of-course tests: 

a) By campus? 

b) By State? 

c) By urban districts? 

1-4. Who in AISD in 1993-94 received 

achievement results from the following 
tests: 

a) TAAS? 

b) NAPT? 

c) ITBS? 

1-5. What questions were asked in 1993-94 by 
the School Board, the Superintendent, 
and central and campus administrators? 
What answer or action was taken with 
each question? 

1-6. What percentage of LEP students were 
not tested with English TAAS because 
they were exempt? 

1-7. How does the percentage of LEP students 
exempted compare to the percentages of 
the other urban districts? 



INFORMATION SOURCES 

• Scholastic Aptitude Test 
(SAT) Report (August 1993) 



End-of-Course File (July 1994) 



End-of-Course File (July 1994) 



• ORE Distribution of 

Reports List (January 1994) 



• ORE Documentation (Ongoing) 



• TAAS File (December 1993) 

(June, August 1994) 

• LANG File (Ongoing) 

• TAAS File (December 1993) 

(June, August 1994) 

• JUEC TAAS Summary (June 1994) 
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Information 
Needs 



Date needed: July 1994 



INFORMATION QUESTIONS INFORMATION SOURCES 

What percentage of students are exempt • TAAS File 

from testing: • NAPT File 

a) On the TAAS? • ITBS File 

b) On the NAPT? 

c) On the ITBS? 



9. How does the percentage exempted on 
TAAS and NAPT compare with other 
urban districts? 



TAAS File 
NAPT File 
JUEC Summary File 



10. How does the number of special 

education students tested or exempted in 
1993-94 compare to the other urban 
districts? 



TAAS File 
NAPT File 
JUEC Summary File 



1 1 . What procedures were set up for • ORE Documentation 

coordinating and administering advanced 
placement tests in 1993-94? Do they 
need to be revised? 
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DISTRIBUTION 
OF 
REPORTS 



Iowa Tests of Basic Skills (ITBS) 
Individual Student Reports 
Elementary, 
Grades 1-2, 
Spring Administration 



All testing materials and reports are sent directly to the principal at eacii school. 



Report 


Number of Copies and Recipient 


Date . J 


Alpha Listing 
Grades 1-2 


1 


Schools 


May 1994 


Percentile Rank Order 
Grades 1-2 

• Reading Total 

• Mathematics Total 


1 


Schools 


May 1994 


Student Score Label (gummed) 
Includes GE and percentiles 


1 


Schools 

(for measurement data card) 


May 1994 


Confidential Student Report 


1 


Schools 
(for parents) 


May 1994 


Fall Classroom Summaries 
This report is generated by 
period, by teacher 


1 


Schools 


Sept. 1993 


Spring Classroom Summaries 
(including ROSE) 


1 


Schools 


May 1994 
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DISTRIBUTION 
OF 
REPORTS 



Norm-referenced Assessment Program for Texas (NAPT) 

Individual Student Reports 
Grades 3-11 
Spring Administration 



All testing materials and reports are sent directly to the principal at each school. 



J Report . .. 


Number of Copies and Recipient 


Date j 


Alpha Listing 
Grades 3-8 
(provided by TEA) 


1 


Schools 


May 1994 


Percentile Rank Order 
Grades 3-8 

• Reading Total 

• Language Total 

• Mathematics Total 


1 


Schools 


May 1994 


Grades 9-11 

• Reading Total 

• Language Total 

• Mathematics Total 








Student Score Label (gummed) 

Includes GTI ana peivczt i, «*^ 
(provided by TEA) 


1 


Schools 

(for measurement data card) 


May 1994 


Confidents J Student Report 

(provi ied by TEA) 


3 


Schools 

(1 for parents/students) 


May 1994 


Alpti* list of eighth graders 
assigned to high school 


1 


High Schools 


April 1994 


Rank-order listing of fifth and sixth 
graders, next year's campuses 


1 


Middle/Junior High School's 


April 1994 


• Reading Total 

• Language Total 

• Mathematics Total 








Rank-order listing of eighth graders 
by next yearns campuses 


1 


High Schools 


April 1994 


• Reading Total 

• Language Total 

• Mathematics Total 
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DISTRIBUTION 
OF 
REPORTS 



NAPT 

Individual Student Reports 
Spring Administration 



All testing materials and reports are sent directly to the principal at each school. 



j Report 


Number of Copies and Recipient 


Date 


Fall Classroom Summaries 
This report is generated by 
period, by teacher. 


1 


Schools 


Sept. 1994 


Spring Classroom Summaries 
(including ROSE) 
This report is generated by 
period, by teacher. 


1 


Schools 


May 1994 


School Summary Report 
(Provided by TEA) 


1 

5 
1 
7 


Schools 

Area/ Assoc. Superintendents 

ORE 

Total 


May 1994 


(Provided by TEA) 

/ 


1 

5 
1 
7 


Final Report/ 

Achievement Profiles 

Area/ Assoc. Superintendents 

ORE 

Total 


June 1994 


School Achievement Profiles 


1 

5 
6 


Schools 

Profiles Volumes 
Total 


July 1994 


* District Achievement Profiles 


20 


Final Report/ 
Profiles Volume 


July 1994 


Districtwide Frequency Distribution 
by Grade by School 


1 


ORE 


May 1994 



* These report! tre not dUtributed to 
ANYONE until the final reports are 
distributed to the School Board. 
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DISTRIBUTION 
OF 
REPORTS 



TAAS 

Individual Student Reports 
Elementary and Secondary, 
Grades 3, 4, 5, 6, 7, 8, and Exit-Level 
Fall and Spring Administration 



All testing materials and reports are sent directly to the principal at each school. 



Report 


Number of Copies and Recipient 


Date 


rnnfirlpniinl Student Resort 
(provided by TEA) 


1 
1 
2 


Students/Parents 
Permanent Folder 
Total 


Dec. 1993 
May 1994 


Student Score Labels (gummed) 
Indicates individual student 
scores, alphabetic listing by 
school by grade for measurement 
data cards (Provided by TEA) 


1 


Schools 

(2 copies for exit-level) 


Dec. 1993 
May 1994 


Alntinhptff* T .Jstinu of Individual 

Student Scores 

This report is delivered by school 
by grade for valid scores 
(produced by TEA) and 
experience only (produced by 
ORE) 


2 


Schools 


Dec. 1993 
May 1994 


Class Summary by Objective 
At the beginning of the year 
by teacher by period (produced 
by ORE) 


1 


Schools 


October 1993 


School Summary Report 
(Provided by TEA) 


3 
5 
1 
9 


Schools 

Area/ Assoc. Superintendents 

ORE 

Total 


Dec. 1993 


* District Summary Report 


1 

5 
1 
7 


Final Report/ 
Achievement Profiles 
Area/ Assoc. Superintendents 
ORE 
Total 


Dec. 1993 


Alphabetic Listing of Students 
Required to take the Exit-Level 
TAAS 

(Produced by ORE prior to 
testing) 


1 


Schools 


Sept. 1993 
February 1993 



* These reporu are not distributed to 
ANYONE until the final report* are 
distributed to the School Board. 
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DISTRIBUTION 
OF 
REPORTS 



Other Tests 
Summary Reports 
Differing Administrations 
Microfiche Production Schedule 

1993-94 



All testing materials and reports are sent directly to the principal at each school. 



I - Report 


Number of Copies and Recipient 


Date 


Customized Reports 

NAFT/ITBS Microfiche 
(Elementary, 
Middle/Junior High, 
Senior High) 


Upon Request 

Districtwide alpha listing of 
individual student scores, 
grades 1-11, including special 
education students tested 
validly or for experience only 


July 1994 
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PROGRAM DESCRIPTION 

In December 1993, the Superintendent and the 
Management Team reviewed a proposed 
allocation of local resources in ORE and 
requested that some evaluation resources be 
devoted to several proposed projects not assigned 
to any ORE staff, including a study of the 
effectiveness of tutorial assistance offered to high 
school students who have not passed the Exit- 
Level TAAS. Students must pass the Exit-Level 
TAAS as part of the requirements to graduate and 
receive a high school diploma. 



TAAS TUTORIAL 



EVALUATION PLAN 



Contact Person: 
Barbara Wiser 



93.07 



Decision 
Question 



Is AISD's tutorial program successfully helping students in need of 
additional instruction to pass the exit-level TAAS? 



Date needed: August 1994 



EVALUATION QUESTION 

1-1. What percentage of students pass the 
TAAS after receiving tutorial 
instruction in: 

a) Writing? 

b) Reading? 

c) Mathematics? 

1-2. What percentage of students pass the 
TAAS if they do not receive tutorial 
instruction in: 

a) Writing? 

b) Reading? 

c) Mathematics? 

1-3. What was the average TAAS score gain 
of students who attended TAAS tutoring 
compared with those who did not attend 
tutoring? 

1-4. What type of instructional strategies were 
most effective? 



INFORMATION SOURCES 

• TAAS File (ongoing) 

• Program Records (May, August 1994) 



• TAAS File (ongoing) 

• Program Records (May, August 1994) 



• TAAS File (ongoing) 

• Program Records (May, August 1994) 



• Program Records (May, August 1994) 
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Information 
Needs 



Date needed: August 1994 



INFORMATION QUESTIONS 

1-1 . Does high school staff direct the students 
to enroll in tutorials by: 

a) Informing students of tutorials? 

b) Encouraging students to enroll in 
* tutorials? 

c) Advising students to enroll in 
tutorials? 

d) Enrolling students automatically? 

1-2. Were the tutorials offered in: 

a) High Schools? 

b) Community Schools? 

c) Evening School? 

1-3. Who taught the tutorials? 

1-4. Were tutorials taught: 

a) During school hours? 

b) After school? 

c) On weekends? 

1-5. How many tutorial sessions were offered 
and how long were the sessions? 

1-6. What percentage of eligible students 
registered to take TAAS tutorials? 

1-7. How are students informed of the 
availability of tutorials? 

1-8. What are the demographic characteristics 
of the students who attend tutorial 
classes? 



INFORMATION SOURCES 

• Staff Interview (May, August 1994) 



• Staff Interview (May, August 1994) 

• Staff Interview (May, August 1994) 

• Staff Interview (May, August 1994) 

/ 

• Administrative Interview (May, August 1994) 

• Program Records (May, August 1994) 

• TAAS File (ongoing) 

• Program Records (May, August 1994) 

• Student Survey (May, August 1994) 

• Program Records (May, August 1994) 

• Student Master File (ongoing) 
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TECHNOLOGY IN 
INSTRUCTION 



EVALUATION PLAN 



Contact Persons: 
Janice Curry 
Melissa Sabatino 



PROGRAM DESCRIPTION 



The District has a long history of applying 
computer technology to instruction, beginning in 
earnest with the District's " computer initiative" in 
the early 1980's. Since the mid-1980's, with the 
assistance of Chapter 1 and Chapter 2 grant 
monies, AISD has installed many different 
computer labs-e.g., Wicat, Writing to Read, 
Computer Curriculum Corporation (CCC), and 
Jostens at the elementary level; and Texas Learning 
Technology Group (TLTG), Technology Learning 
Center, and CCC at the secondary level. Other 
major sources of funding for computer technology 
in AISD include grants from IBM, Apple, the 
Texas Education Agency, Pepsi Corporation, and 
RJR Nabisco, District funds, and Parent-Teacher 
Association resources. In 1993-94, AISD employs 
computers for instructional purposes extensively 
throughout the District. 

Local resources have been allocated to evaluating 
instructional technology in District schools, and 
Chapter 1 resources have been allocated to 
evaluating instructional technology in Chapter 1 
schools. The resources will be used to determine 
what technology models exist in the District and to 
evaluate the effectiveness of integrated learning 
systems (CCC and Jostens) at schools which have 
utilized these systems for one or more years. 
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Decision 
Question 
J 



How effective were integrated learning systems-Computer Curriculum 
Corporation (CCC) and Jostens Learning--at AISD schools having them 
for one year? 



Date Needed: June 1994 



1-1. 



EVALUATION QUESTIONS 

How many students were served by an 
integrated learning system? 



INFORMATION SOURCES 

• Record of Students Service (ROSS) Forms 
(Ongoing) 

• Computer Log Data (Ongoing) 



Did students served by CCC or Jostens 
achieve higher gains than similar 
students who did not receive these 
services? 



• ROSS Forms/Computer Log Data 
(Ongoing) 

• ITBS/NAPT File (April 1994) 

• TAAS File (May 1994) 



1-3. What was the rate of promotion for • Student Master File (May) 

students using Jostens or CCC systems 
compared to the District promotion 
rate? 



1-4. How cost-effective are the CCC and 
Jostens systems compared with other 
supplementary programs? 



1-5. Are the integrated learning systems 

utilized in a lab setting or as distributive 
networks? 



Student Master File (Ongoing) 
Achievement Data (Ongoing) 
Financial Records (Ongoing) 
ROSS Forms (Ongoing) 

Staff Interviews/Surveys (March) 



1-6. Is there a difference in the effectiveness 
of the systems used in a lab setting 
compared with their use as a 
distributive network in the classrooms? 



• Staff Interviews/Surveys (March) 

• ITBS/NAPT File (April) 

• Observations (March/April) 



1-7. To what extent do teachers utilize the 
capabilities of the integrated learning 
software? 



Teacher Survey (March) 
Observations (March/April) 
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Decision 
Question 1 
Continued 



EVALUATION QUESTIONS 

1-8. Has there been an increase in the 

development of higher-order thinking 
skills for students using integrated 
learning systems? 



INFORMATION SOURCES 
• NAPT File (April) 



1-9. Are students on task while engaged 
with the computer? 



1-10. What is the role of the principal in the 
implementation of the CCC or Jostens 
systems? 



• Computer Log Data (Ongoing) 

• Observations (March/April) 

• Teacher Survey (March) 

• Teacher Survey (March) 



1-11. 



How many minutes per week are 
students using the integrated software? 



• Computer Log Data (Ongoing) 

• ROSS Forms (Ongoing) 



1-12. To what extent are teachers integrating 
the computer software with their 
classroom instruction? 



• Observations (March/April) 

• Teacher Survey (March) 
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Information 
Needs 



What technology models currently exist in the District and in which 
schools? 



Date Needed: June 1994 



INFORMATION QUESTIONS 

What computer hardware is being used 
for instructional purposes at each school? 



INFORMATION SOURCES 

• Administrator Interviews (March 1994) 

• Technology Inventory (February 1994) 



Are the computers and other instructional 
technology utilized in a lab setting or as 
distributive networks in the classrooms? 



Administrator Interviews (March) 
Technology Inventory (February) 



What types of instructional software are 
used in each school? 



Administrator Interviews (March) 
Technology Inventory (February) 



What was the major funding source for 
the instructional technology equipment 
and software (e.g., local District funds, 
Chapter 1 funds, Chapter 2 funds, 
grants, or PTA funds)? 



Administrator Interviews (March) 
Technology Inventory (February) 
Financial Records (Ongoing) 



Who assists students with computer- 
assisted instruction (i.e., lab tech, aide, 
classroom teacher, volunteer)? 



Administrator Interviews (March) 
Teacher Survey (March) 
Observations (March/April) 



What is the usage level of the Windows 
on Science laser discs? 



Teacher Survey (March) 



How much time per week (on the 
average) did students spend on the 
computers by: 

• Grade? 

• Achievement Level? 

• Sex? 

• Ethnicity? 



Laboratory Sign-up Sheets (Ongoing) 
Teacher Survey (March) 
Computer Log Data (Ongoing) 
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ATTACHMENT B 



REQUESTS FOR ORE LOCAL STAFF RESOURCES 



School Support 


Activity 


Description 




FTEs 
Rsquirad 


FTEs 


UaitonWfth 

Aieea/Schcois/Committees 


Attand and mtka praaantatkina at araa meetings. Prepare 
graphic mat a rial*. 


-• A . 




0 60 : 


DISC Profit™ 


Coordinate the production and mailing of school profiles 
twice e year. 




0.26 


0.26 


Technical Assistance 


Respond to schools' requests for technical ss si stance. 


' : A 


0 50 


0.50 


AccountebHrtv/Support 


Provide/Interpret deta for schools. 


A 


1.66 


1.56 


Achievement Tatting 


Coordinate, process, end report results of fTBS, NAPT, 
TAAS. 


. BT 

■''s ' 

A 


2,76 




Advanced Placement/Credit 
by Exam 


Function ss lieieon with TEA. Oisseminete requirements end 
regulations in AISD. 


S 


0,10 


0.06 


Identifying At-Risk and 
Dropouts 


Produce " at-risk" and dropout rosters and reports. 


BT 

S 

,:■ A . 


■ : :;. 'Q,60 : 


0.46 


School Ganaratad Survaya 


1 Assist schools in designing and processing surveys. 

i 


A 


0,26 




Total School Support 


6,50 


6.16 



Mandate: BT * Board of Trustees; S -» State Requirement; A * Administrators/Superintendent 
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ATTACHMENT B 



REQUESTS FOR ORE LOCAL STAFF RESOURCES 



Evaluation 


Activity 


Description 




FTEe 
Raqurad 


FTEe 


Bilingual Program (IBP) 


Evekiate bilingual education and English ss a Second 
Language programs for limited-English-profJcient student; 


S 

a ; 


0.30 


0 30 


Program Cost Effectiveness 


Asaeae cost-effectiveness of 1992-93 special programs 


BT 

;,A- , 


0.3$ 


0.35 


DO Prevention for 
Overage/AIUGED 


Evaluate eras programs for dropout prevention among 
overage 9th graders; evaluete contracted dropout recovery 
programs: AIL, RAYS 


*: ' 


t'00 


0. 30 


Sanchez - Year - Round 
School 


Evaluete year-round school concept implemented at 
Sanchez Elementary 


■ A :- 




: ft- OK 


CI ■■■room Observations 


Conduct day-long observstions of individual students to 


•■ A" 


.0.30 




Program Overlap Study 


Study of overlapping services among 1993-94 special 
program* 


A 


0.10 




DISC Phase II 


Coordinate next phase of Distributive informetion Systems 
to Cempusee project to decentralize date-access and 
information-generating cspabil'rties 


8T 

s 

, A 


1.00 




Mentor Program 


Evaluate the District's mentor program 


A 


0.30 




Technology in non-Chapter 
1 Schools 


Evekiate the effect on student achievement of introducing 
new technology in non-Chepter 1 schools 


A 


- I*W 


: O BO 


Tutorial! for TAAS 


Evaluate the effect of tutorials on student TAAS 
performance 


A 


0.50 




Vocational Education 


Eveluste the effectiveness of new school-to-work transition 
programs 


A 


0*10 




SB! Process 
Evaluation/Wei vera 


Evekiate the effectiveness of District efforts to move to 
school- be ted management; evsluste effect of w livers 
requested by schools 


A. 


1 00 




Writ inn 1 Mhm 
\nriTung Laos 


Evaluate the effectiveness of writing lebs in improving 
student writing achievement 


A • :, 


.\.,t.sa 




Homework Hotline 


Evaluate the effectiveness of offering students sssistence 
with homework vie hotline 




0.30 




Staff Development 


Evaluste the effectiveness of distrtctwide stiff development 
prefects through the Staff Development Academy 


A 


. 1,50 




Parsnt Involvement 


Evaluste the effectiveness of dlstrictwWe parental 
involvement activities 


A 


1.00 




| Total Evaluation 




| 10.50 


j 1.90 



Mandate: BT = Board of Trustees; S * State Requirement; A » Administrators/Superintendent 
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ATTACHMENT B 



REQUESTS FOR ORE LOCAL STAFF RESOURCES 



Research and Analyses 


Activity 


Description 


Midi! 


PTEs 
fl«qur«d 


FTEs 


GENESYS 


Coordinate the collection of dais files and GENESYS 
production. 


" *:;>..; 




<M0: 


Coordinated Survey 


Collect survey items, plan sampling, produce, mail, scan, 
and report results of survey. 


1 - A: 






Sc! ool Oimata Survey 
(Disign) 


Design (.05), produce, administer, process, and report 
results of survey. 


: , A|*v- 


.a as 




Fonner Student Survey 


Design, produce, administer, process, and report results of 
survey. 


BT 

=. ■ ' : A- 






Historical Retention 


Analyze end report retention dete. 


' S . " " 

A- 


0,25 


0,15 


Feculty/Steff 


Analyze and report AISD personnel dete. 


BT 

:. A . 


0.05 




Student Mobility 


Analyze and report student mobility data. 


A 


^10 


0.06 


Technical Attictence/Ad 
Hoc Requeat 


Respond to requests from central administration and outside 
A ISO agante. 


: A ly.;' 




;;';ll.0O .; 


CnnrH In attnn of PvtAmsi 
Research 


Coordinate the review and spprovel process of eppiicetione 
for permission to conduct research in AISD by external 
agenciee and individuals 




a. to 


0,10 














































v ':. 






1 Total - Reeearch and Analyse 




2.86 


Z20 



Mandate: BT » Board of Trustees; S - State Requirement; A - Administrators/Superintendent 
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ATTACHMENT B 



REQUESTS FOR ORE LOCAL STAFF RESOURCES 



I Other Activities _ 


j Activity 


Description 


FTEs 
R«qur«d 


FTEs 


I Supervision 


Supervision of staff. 


:o.46 


0.45 


MeneaorieJ/Offlce 


Secretarial end officewide dutbs. 


f " ! -; : 2,90'. : - 


2.90 


j Total- OtrwAotivfctet.. 




3 36 


3 36 



SUMMARY 


Description 


FTEs 
Required 


FTEs 


School Support 


6.50 


6.15 


Evaluation 


10.50 


1.90 


Research and Analyses 


2.86 


2.20 


Other Activities 


3:36 


3.35 


Total - FTEs Requfred/Allocated 


23.20 


13.60 



Full Time Equivelente {FTE) 



[— 


.06 


.10 


.20 


.30 


.40 


.60 


, .60 


.70 


.80 


.90 


1.00 


.tt Hours per 
U Yeer 


92 


184 


368 


552 


736 


920 


1104 


1288 


1472 


1656 


1840 


■ Hours per 
| Month 


7.7 


15.3 


30.7 


46.0 


61.3 


76.7 


92.0 


107.3 


122.7 


138.0 


163.3 


A Hours per' 
| Week 


2.0 


4.0 


8.0 


12.0 


16.0 


20.0 


24.0 


28.0 


32.0 


36.0 


40.0 
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Overage Ninth-Grade Students 
Dropout Prevention Programs 
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Extended School Time 
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Title VII Bilingual Transition Program 






» 
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Title II Mathematics and Science Teacher Training 
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s 
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s 




National oraancB raunvminjn 
(Science Academy ol Austin) 






S 


























Drug-Free Schools Program 






s 
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Programs for LEP Students 
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Program Effectiveness Comparisons 
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AIL Dropout Recovery Program 


s 






s 


s 




S 
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78702 RAYS Dropout Recovery Program 


s 






s 
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s 


s 


TJ 


Technology for Instruction 








s 


s 
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s 


ii 


CZanarir Evaluation cjudom fftFNFSYS)* 
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Historical Retention in Grade 
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Former Student Survey 


s 
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TAAS Tutorials 
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% 








►up port 


Distributive Information Systems for Campuses 
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Schools 


Student Achievement 
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s 


s 




s 


s 


s 


s 


s 


s 


s 


s 


s 


s 


s 




o> 
.£ o) 

_ 






2 

1 

* 


o- 






JO 
VI 
tt 
O 

Q_ 


UJ 

tr 
O 

« 

E 
E 

§ 

CC 


■■8 
s 

CL 


UJ 

cc 
O 

C 

a 

6 
E 

§ 

CC 




&• 

IS 

f 

CO 


to 

c 

fl 

1 






ORE 
Agenda-Se 
Processing 1 


-o 
to 
& 

l 
S 

3 

? 

1 

"3 
a- 
a 

a: 


o» 
*a 

3 
UJ 

8 

CO 

o 
« 

CO 

? 

"a 

s* 

cc 


1 
to 

? 

C 
3 

*ti 
c 
* 

9 

'5 
? 

cc 


c 
.9 

e 

c 

*E 

1 

c 

9 

"O 

c 
o 

.£ 

1 

3 

CT 
« 

cc 


«! 

8 
3 

c 
* 

Q. 

• 

c 
Q 

.2 

-o 
>. 

• 

,CC 


UJ 

cc 

o 

>s 

1 

« 

I 

c 
2 
po 

3 

> 
Ul 


( 
C 


IS 
•1 

11 


• 

o 

.8 
> 

3 <9 


Does it provide new/useful information? 


Is there potential for findings being used? 
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